Workforce

A small federal agency plans to document the state of American museums

The Institute of Museum and Library Services, a small, independent federal agency, plans to examine as many museums in the U.S. as it can.

Eric White@FEDERALNEWSCAST

March 12, 2024 1:55 pm

11 min read

The Institute of Museum and Library Services, a small, independent federal agency, plans to examine as many museums in the U.S. as it can. It’s National Museum Survey will launch in 2025. The agency recently completed a pilot effort to test its survey process. For details, Federal News Network’s Eric White spoke with Laura Huerta Migus, deputy director for Museum Services at IMLS. And with Matt Birnbaum, the agency’s Director of Research and Evaluation on the Federal Drive with Tom Temin.

https://www.imls.gov/news/institute-museum-and-library-services-announces-plans-launch-national-museum-survey-early-2025

Interview Transcript:

Laura Huerta Migus So the National Museum survey is the culmination of more than a decade of work here at the Institute of Museum and Library Services to create a federal data set on the presence and reach of museums across the country. And I think something important to share when we say museum is that at IMLS, we actually have a very inclusive definition of museum that includes art museums, history museums, but also other disciplines like science museums and science centers, zoos, aquaria, botanical gardens, and other types of cultural institutions. And so this effort is building on previous efforts led by my colleague, Dr. Matt Birnbaum’s office in our Office of Research and Evaluation, to create a listing of museums. And this effort has updated that and then also has given us information on how to best get information about museums operation and reach on a regular basis.

Join us May 6 at 1 p.m. EST for Federal News Network's Industry Exchange Data where we'll explore how you can employ data to modernize your agency. | Register today!

Eric White Yeah. Matt, what role did you and your office have in this and how did you go about it?

Matt Birnbaum Yeah. Thanks. It’s been a wonderful partnership where we’ve tried to blend the expertise of my office in this type of applied social science from the survey and statistical methodologies, and blend it with the expertise of those who come from the leaders and others from the museum sector, to make a some type of survey that would be relevant. From our end, thinking about a survey, there are three pieces to it. You need a list of people and institutions that you want to survey that we’re going to call that the technical term is population frame a universe. And right now, we had to come up with an estimate that we could stand by that can be validated just the way any census study is. And right now we’re estimating at about 20,000 plus museums. The second part is we need to come up with a questionnaire. The questions that would be in the survey and part of the whole process was using a whole series of methods and processes in this type of work to come up with a questionnaire. And then you need a mode. A way to administer that survey could be telephone, could be email, it could be personal physical mail. All these things were all had wide variations and issues to consider. And we spent several years in working Laura’s office, my office, some of the leaders across the museum sector working with us as subject matter experts and then partners from the American Institute of Research in developing the plan and then running seven experiments last summer to pilot the survey.

Eric White Yeah, I want to just get a little bit into the methodology there. So how do you even figure out who to send it to? And when you did, were they more than happy to answer your questions? I mean, as somebody who’s been to your local museum, usually the people who run there are usually pretty happy to give you the kinds of information that I imagine you’re looking for. How did that process go?

Matt Birnbaum Sure. There’s a lot there. So I’ll just take the first part about who do you survey and what counts as a museum. So we had a definition, and it’s going to be a nonprofit entity. It’s going to cover these wide range of disciplines, those museums that are kind of living at artifacts and those that can serve other types of artifacts. It’s going to be open to the public 120 days the year. It’s going to have a physical location. From that, we began discussions with the Department of Census and some others, and we ascertained that we weren’t going to be able to get the data that we needed of these museums in a way that we could use it. So we entered into an agreement with Yelp to crowdsource this data, and we purchased a data set from them, and it had over 100,000 records. So at that point, I’ve got some really smart people, really intelligent, and they went to work behind the scenes using all the best data science of cleaning the data using traditional techniques of triangulating and fortifying missing data, moving to some of the most advanced pieces of data science out there, including using ChatGPT. It’s one point for identifying missing websites, using Amazon’s Mturk for trying to think about email addresses. All these types of techniques for coming up.

Matt Birnbaum So we ultimately were able to take Yelp’s database with another data set that represented some members of the museum association. We were able to come up with right now an estimate about 20,000 museums that fit. So that’s the first piece of that. Then you had a second piece of that that’s running concurrent, where we’re working really closely on Laura’s leadership and some of her team and my team providing support with these wonderful leaders from the museum sector to develop a questionnaire. And how you go about and do that is you start initially with what we going to use the term of respondent research. Where we did some initial surveys and some focus groups about what would be the types of survey or research that would make a difference, that would be meaningful, that we had the evidence. You can actually use it to help inform a conversation. And so that was going on at that point when we were putting together the whole vision for the survey. From that, we worked with our subject matter experts and some others, and starting to think about what the questionnaire was. We then went back out in the field again and did some focus groups, cognitive interviews to really start ground truthing this questionnaire. And we got it to the point where probably early last spring, we reached out to OMB and said, hey, we would like to be able to go ahead and test this pilot, and we’re going to test this pilot. I’m trying to run this questionnaire seven different ways using randomized controlled experiments to start learning about which of the ways were most effective. And we ran it probably the worst time of the year, you can think of it rather the dead of August, right before Labor Day when people are on holidays. But to my mind, we got a really wonderful for that first baseline response, which it’s pretty good is about 17%. And it gave us a lot of insights into over the next year as we get ready for the next annual data collection. What are some of the additional revisions we need to do in the questionnaire, in the refining our population frame, and just thinking about a little bit about how we engage with others in the museum sector.

Eric White And so, Laura, when it comes to this data, and obviously this is a big project that Matt just described. When you go to your the folks that you report to in the administration, how do you sell this pilot effort? And what do you say that you’re going to get out of it if they give the all clear to it and give you the green light.

Laura Huerta Migus That’s a great question, Eric. I think the most important selling point is that most folks actually think that this data already exists. And so they’re shocked that a data set like this on the presence of museums doesn’t exist already. And I think that says a lot about how people value museums and already think about them as an important system. And so it’s a pretty easy sell that we’re filling the gap for a set of data that people really just expect for there to be already. I would say, additionally, there are a couple of key facets to our approach in use for this future data set. And the first is that the data is going to be free and publicly available. There are a number of surveys out there for museums and in the museum world, but most of them are either pay to play or behind some kind of a paywall. And from the practitioner perspective, this is a really important way for us to fulfill our service to the museum and library and policy community by creating a rigorous and freely available data set on museums. So that’s one of those selling points. And the other is that it will help us, as the federal government’s largest funder of museums, really quantify the impact of the investments that we make in the museum field, but should also serve as an important tool for our related, funding organizations and cultural organizations that also fund museums so that we can start to quantify the needs, as well as the value of the investments that the federal government and other kinds of funders are making in museums. And we don’t have that kind of universal benchmarking right now. So that’s a real need that we’re filling with this effort.

Eric White Yeah, it seems as if this is an effort that is looking to benefit the consumer. But also folks like you who are, you know, in the monitoring of the museum business. Are there any other uses that or people who could use this data set, that may not be the first people you think about coming to mind?

Read more: Workforce

Laura Huerta Migus Sure. So I actually think about city and county leaders, like mayors and county commissioner boards that are thinking about cultural and community development, and they want to benchmark themselves against other communities and their vitality. And this data set is the kind of information that they could have at their fingertips. What is the presence of libraries, museums and performing arts is very much a conversation that happens at the local level. And right now there’s a gap around museums and how they fit into the economic fabric at the local level. I also think that there’s great potential for the general public. There is not a place where a person can go and really find out where are museums all over the country, even in their own communities. Matt talked about the importance of directories like Yelp. We know that they’re not perfect and those are crowdsourced. And this could really drive those kinds directories for the general public who we know love to visit museums.

Eric White All right, Matt, tell me what the next steps are and what you all are hoping to continue to learn and expand this project as it becomes a reality for you.

Matt Birnbaum I want to follow up on what Laura just talked about. We don’t know enough yet about how our audiences can best utilize the data. It could be that just giving them that data and allowing them to do their own statistical permutations will suffice. It could be that having us, or someone who we commissioned to do a full research piece that talks about some of the critical themes coming from that, that’s going to be the most valuable input. It could be that creating tools online per se, helping them make comparisons of one type of museum to another, having that type of querying capacity for them, that would be the most important. So one of the things that we’re going to be spending this year on is starting to doing that development, starting to work with audiences and starting to think about how we build that, because the added value of this is going to be having this data in a way that’s meaningful and actionable for others. While that’s continuing, we’ve got a lot of lessons to learn from the piloting. There are some areas of the questionnaire that will be wanting to refine, do some additional revisions, interviews, focus groups. We’re going to be having to update the population frame, get it really quite precise. And then because this will be part of the Federal statistical system, there’ll be a conversation with OMB related to the Paper Reduction Act to make sure that if, as we’re planning to go scheduling beginning this annual collection in about a year from now, that the benefits from collecting this information far outsee any burdens we’re going to be placing on people to participate in it.