Long-term Impacts of Early Childhood STEM Instruction

One of the projects I’m currently working on is a formative evaluation for a multi-year grant with PBS Kids to develop STEM-oriented programming for young children. Its not really possible to see longer-term impacts of instructional programming (esp. with a formative evaluation initiative – focused on quickly gathering feedback to help drive immediate changes/improvements to a product), so its exciting when other researchers are unable to uncover links. On that note, I’m excited to share a link to the following article which highlights recent finding that connect early STEM learning to outcomes observed later in a child’s life.



IRB deregulation on the horizon!

At long last, we can see some new Human Subjects guidelines/regulations on the horizon! An article entitled “Long-Sought Research Deregulation Is Upon Us. Don’t Squander the Moment,” recently appearing in The Chronicle of Higher Education, provides more info on the upcoming changes: https://shar.es/1F3hQF

In the meantime (i.e., til January 2018), if you are doing work with Human Subjects and/or have been asked to get an IRB review – this webinar may be of interest:

“Surviving the IRB Process” (part of a professional development partnership between VSA and ASTC)

Upcoming visitor studies training opportunities

I’m really excited to announce a couple professional development opportunities that I’ve been helping to organize on behalf of the Visitor Studies Association, in my capacity as chair of the professional development Association.

The first is a webinar on April 27th featuring presenter, Beverly Serrell–the woman who literally wrote the book on Tracking and Timing as a method for studying visitors in exhibits. This is part of a series of professional development opportunities that VSA is co-hosting with ASTC: http://www.astc.org/profdev/webinar-tracking-timing/
Webinar_Timing-Graphs.jpg    Serrellheadshot copy
Registration link: https://members.astc.org/ASTC_Prod_iMIS/EventDetail?EventKey=PD2017VSA1&WebsiteKey=56a4f481-6bb9-45a8-96a1-f9abf9c90cbc

The other event is the 2017 VSA conference in Columbus Ohio.  We’ve got a lot of great Pre-conference workshops planned so be sure to check out all of these great PD opportunities as well: http://www.visitorstudies.org/conference-registratio

Screen Shot 2017-04-18 at 10.57.46 AM.png

Election-themed babyshower

This week we celebrated the upcoming arrival of Alison’s baby with an election-themed baby shower. With a due-date near the US elections in November, this seemed like a fun way to forego traditional pastels for some good old red, white, and blue! Alison actually inspired this theme when she sent an election themed package to let us know that she was expecting. Congrats Alison – we can’t wait to meet baby Allen!

The Data Revolution

Screen Shot 2016-08-02 at 9.47.56 AM.png

This year’s theme for the Visitor Studies Conference–playing on the historic revolutions that took place in Boston, where this year’s conference was held–was “The Data Revolution.”  There were an impressive number of presentations and workshops that focused on data and a variety of approaches to data analysis.  As always, I’m happy to share out my conference notes, as well as link to my presentation (along with co-presenters Claire Quimby and Elee Wood) “How to Keep from Drowning in Data.”  Lastly, I hosted a dining discussion on the topic of “Cool New Tools, Tips, and Tricks for Evaluation.”

Below (an example from my notes): a scattergram that plots several data points from different exhibits based on Sweep Rate Index (SRI) and the Percentage of Diligent Visitors (%DV)


Less is the New “More”

Big is out – and tiny is in!


If you haven’t been hiding under a rock recently, there’s a good chance that you’ve heard of the tiny house movement. This trend toward small and ultra-small houses has been gaining momentum worldwide as more and more people have been seeking simple and affordable housing options.

One of the underlying reasons is that people have embraced the notion that having more stuff doesn’t necessarily make people happier.  Perhaps the same can be said of reports.

Over the course of my career as an evaluator, I’ve written many reports that have exceeded 100 pages – but rather than being the useful resources that we hope they will be for our clients, they often become glorified bookends. Maybe less is more where reports are concerned as well.

Can we go from this:


to this:



Okay, not literally tiny reports.  But seriously, I’ve been sensing a growing interest in and demand for tiny reports – i.e., reports that total less than 20 pages and take less than a half hour to read – Reports that give readers a good sense of the key findings without inundating them in every detail of an extensive study – Reports that are far more likely to be read, shared, and referenced than their lengthier counterparts.

I’m not sure that all clients/evaluation stakeholders are ready to embrace the notion that less is more – and that might be the first challenge: i.e., getting the primary audience to see how a shorter report might be worth a great deal more than a much longer/more comprehensive report.

An equally daunting challenge–once you’ve gained buy-in for the idea of producing a tiny report–is the fact that its not always easier to write less.  Presenting information in the most succinct/elegant way possible can actually be more time consuming and mentally taxing than mindlessly spewing out every single finding. Despite these challenges, however, I’d argue that the resulting use of a tiny report should be the driving factor in advocating for, and adopting, the move toward shorter reports in contrast to longer more burdensome reporting formats. After all, a great report is only truly great if it gets read and used – no matter its size.


Updated 5/31/17:

I loved this related post by Kylie Hutchinson on the AEA 365 blog – “The Demise of the Lengthy Report.” I especially appreciated the hamburger analogy and image – and it serves as a great reminder that its always important to check stakeholders’ appetites where reporting is concerned.


It depends…

“It depends” is a phrase that most evaluators know well.  In the work that we do, context and participant characteristics often matter a great deal.  In this recent EdSurge article by Patricia Gomes she explores concepts and caveats that relate to efficacy as discussed by Barbara Means and Jeremy Roschelle, co-directors of SRI’s Center for Technology in Learning:


Big takeaways:

  1. Tools don’t exist in isolation.  How they are used by educators matters a great deal. A great tool being implemented half-heartedly or incorrectly may not produce desired results.  Conversely, a poor tool being implemented in the right context by a skilled educator could produce results that wouldn’t be seen in other contexts.
  2. Educators often seek a one-sized-fits all approach, but like cars, maybe the best product varies based on the needs and preferences of different users.
  3. Measures matter too.  You’ve got to know what you are looking for and pick measures that can effectively identify desired outcomes.  (I would also add a note about the fact that it is also good to incorporate methodology that also allows you to uncover unintended outcomes – if we are too focused on very specific outcomes, we may miss other unanticipated outcomes–both positive or negative–that could also be important to understand).
  4. Some outcomes may take time to emerge. Big and fundamental changes in learners take time.
  5. For all these reasons: “It depends” is a valid and accurate response for people who wonder if certain ed tech tools work.