Notes from the American Evaluation Association’s Conference (Eval2023)
Uncategorized
Its all about partnerships: Lessons learned at the Indiana Summit on Out-of-School Learning
I had the chance to attend and present at the Indiana Summit on Out-of-School Learning held in Noblesville on April 25th and 26th and had a wonderful time connecting with colleagues doing great evaluation and programming work in the out of school sector. It also provided an excellent overview of current funding opportunities and other trends that are currently impacting this important sector within the larger field of education.
National Afterschool Association 2023 Conference
Unlike the last NAA conference association which seemed to have a prevalent undertone all about getting others within education to take the role of afterschool educators more seriously, this post-COVID event was focused more on dealing with the lingering challenges of the pandemic, with burnout and staffing challenges being among the most common themes.
I’ve bundled together my notes from this year’s conference and made them available here. I’ve shared a few pages below as a teaser.
GDC, we meet again
After a 10+ year hiatus from attending the Game Developers Conference, I am happy to report that I had the opportunity to attend again this year…and in person nonetheless! For contextual purposes: I started going to GDC (back when it was the Computer Game Developers Conference) in the mid-nineties. At that point, I was more actively engaged in game design and interactive media design work as part of my grad school program – so I was thrilled to get to serve as a Conference Associate. For those who aren’t familiar with the CA program at GDC I will put in a quick plug for it as one of the most amazing volunteer programs I’ve ever experienced and note that it offers access to an amazing network of people who are a part of (or seek to be a part of) the game design industry.
I wasn’t able to make it to GDC every year, but I tried to go every few years to keep tabs on what all was new. Over the better part of the past three decades, it has been amazing to see the industry change and to get to witness the rise (and sometimes fall) of different trends.However, when my worked veered more toward other modes of educational program and product research that wasn’t exclusively focused on games, I stopped attending on a regular basis to make room for others seeking to serve as Conference Associates. Then there was this little pandemic thing that happened, and the organizers of the CA program made an impassioned plead to former CAs for help because they were having a hard time getting enough applicants that were willing or able to attend this year’s event in person…so I figured it was time to give it another go.
My interests still lie in educational games and educational games research so that’s where I tried to focus my attention at this year’s conference. The educational games landscape at GDC looks a little different than when I last attended in 2007 and, overall, the industry looks a lot different than it was at its heyday in the later part of the 20th and first part of the 21st century. Nonetheless I was happy to find that there were a few sessions that relate to the work that I currently do (e.g., usability testing and games-based research and evaluation). Attached are a few of my session notes.
Notes from “Scaling Impact”
The Indiana Evaluation Association had just begun planning it’s quarterly meeting when the coronavirus was presumably hitting the shores of the United States. Thankfully, a can-do attitude prevailed and the group was able to pivot to an online session rather than the usual face-to-face gathering and the desired presenter, John Gargani was game for participating remotely.
The session, held this past Friday, was both a masterful overview of topics and concepts presented in John’s book “Scaling Impact: Innovation for the Public Good” – co-authored with Robert McLean – and also an excellent example of a remote professional development session.
The following are a few of the key takeaways that I came away with:
- We assume impacts will be positive, but as evaluators we should come at them from a more neutral perspective and be prepared to consider those that are not positive, as well as those that were unanticipated.
- Where scaling is concerned, more isn’t always better, and better isn’t always better. Scaling is a choice. The context for scaling and various tradeoffs must be considered, e.g., the impact risks of scaling (lower vs. higher risks associated with scaling — or not scaling and the level of certainty that desired impacts will occur).
- The choice to scale should be shared by those who will be affected. We need to start by asking “Should we scale?” rather than “how can we scale?”
- Scaling is itself and intervention. Ultimately, the success of the intervention should be measured by the ultimate outcome, i.e., the result, rather than whether the scaling method first selected was successful in bringing about that result. It is the end that matters more than the means.
Better ways to work from home
I’ve found that I’ve liked working from home. Quite a bit in fact. I don’t miss the commute back and forth. I don’t miss the monotony of sitting at the same desk all day (I have a couple spots that I’ve found at home where I feel productive for different types of tasks/at different times of the day). I do miss my colleagues, but I’ve really enjoyed getting to interact with them in new ways online. I also like getting to see my husband a lot more during the work week.
All that being said, I know that I’m probably in a minority as one of the happy few among the millions of people who are more frustrated and challenged by the transition to working from home. However, even though I feel that I’m doing pretty well on the working-from-home front, it doesn’t mean that there aren’t new things I can learn. To that end, I jumped on the chance to attend an online conference on “Remote Work” at the tail end of March. The conference was held over three days, and due to my schedule (which now extends a little bit into most evenings so that I can better coordinate work efforts with my colleagues working on the West Coast), I was only able to actively participate in two of the sessions that were held, but I still came away with a fair amount of new-found knowledge on how best to foster positive work-at-home experiences.
Quick Tips and Helpful Eval Treasures
Compiled for and by members of the American Evaluation Association:
New Year, New Motto
There are times when the purpose of evaluation seems to get lost in the shuffle in a rush to get a program off the ground or a product out the door. In those cases, evaluation can feel like a necessary evil rather than an important step of the overall process.
I wanted to start the year off with a motto that could help me remember the importance of what I do, so that I can help clients (and potential clients) see the inherent value of evaluation as well. In my quest for that motto, I stumbled across this oldie but goodie: “That which is measured improves” – rumored to have been said by Karl Pearson a statistician. Sometimes referred to as “Pearson’s Law” I think this quote sums up one of the important benefits of evaluation – i.e., its potential for making something even better.
In the absence of measurement, we arguably don’t know if there has been improvement. As the old “G.I. Joe” saying goes, “knowing is half the battle,” but, I like the implied assertion in Pearson’s Law that the act of measurement may foster and help to facilitate improvement. That’s what I seek to do as an evaluator: help make everything the best it can be!
Reflections on Evaluation 2017 Conference
Just wrapped up a great week in DC for Evaluation 2017. I met up with many of my friends and colleagues from the visitor studies world; got to spend time with my fellow Rockman et al team member, Kristin Bass, and reconnect with former Rockman team member, Maryann Durland; met with some former and new clients for the first time face to face; gave a presentation on telepresent data collection and hosted a roundtable discussion on assessing the impacts of STEM media projects for youth; and even got to hang out with Michael Quinn Patton in the very last session that I attended…if there is such a thing as “evaluation rockstars,” Patton should definitely count among their ranks, and I’m certainly a big fan of his utilization-focused approach to evaluation. So, I’d say it was a pretty good week! 
As always, I like to share my notes so that others can also have a chance to benefit from all the things I was able to hear and learn at this week’s conference. I also find conferences to be a great opportunity to hone my note taking skills.
Without further ado, here’s the link to my notes: https://cl.ly/1h263S1V3u19
Your Eval Pal 