Receiving grant funds to implement an innovative initiative can both be exhilarating and nerve-wracking. The usual responsibilities—and self-doubts—can intensify as the annual report deadline approaches. How do you start? What details do you include? In the Rucks Group’s Annual Report webinar, you will learn what most funders, and the National Science Foundation in particular, want to know from grantees. Our free webinar covers:
Welcome, everyone, to the Get a Grip! Tips to Ease Annual Report Anxiety, our next installment in our coffee break webinar series. If you have been within the grant space for very long, then you will understand this distinction between what occurs in the planning phase (how you have everything laid out and how things will be going when you’re planning out a grant) versus what it’s like at implementation and when you’re actually implementing. I think in the very same vein, that occurs as well when thinking about your emotional response when you found out that you won a grant versus your emotional response when you have to complete the annual report. With that as context, what we wanted to be able to do today is to share some information to help reduce that anxiety and some of that dread around annual reporting.
Specifically, what we’re hoping to do is: provide some context around what the basic elements of an annual report are – I think providing a conceptual framework is really helpful in terms of how to approach an annual report; then we want to be able to share out some tips and best practices in completing your annual report; and then, of course, we want to make sure that we’re answering your questions. To be able to answer your questions, just make sure to use the question function on your computer. And of course, we have with us Alyce Hopes, our Outreach Coordinator who will help facilitate the Q & A that’s sprinkled throughout this Coffee Break Webinar. I should introduce myself as well. I’m Lana Rucks, Principal Consultant of The Rucks Group. The Rucks Group is a research and program evaluation firm that gathers, analyzes, and interprets data to enable our clients to measure the impact of their work. We were formed in 2008, and over the past several years, we’ve worked primarily with higher education institutions and grants funded by federal agencies such as the National Science Foundation (NSF), the Department of Education, and the Department of Labor. With that as background, let’s get started by first talking about the basic elements of an annual report.
Again, as a little bit of context, last summer we had a Coffee Break Webinar series; it was a three-part formula for writing a grant proposal evaluation. Throughout that series, one of the elements of the argument was that there were just basic components of an evaluation report. You can think about it in terms of if you have a small-scale grant, then you push in the elements. With a large-scale grant, you pull out the elements – kind of like an accordion. So small-scale, you have slightly less detail and slightly less information, and then with a large-scale, you pull out the elements you have slightly more detail that’s provided. Having that type of frame helps you in terms of knowing how to approach writing an evaluation plan. I pull that in as context because I think that the same idea of there being basic elements of an evaluation plan applies to the basic elements of an annual Report. I think as you understand that in annual reports, although they may be using different terminology or presenting thins in different ways, in essence, there are only so many different elements they’re really asking about – this is what I think those elements include: first I think it’s going to ask, “What did you say you were going to do?”.Then it’s going to ask, “What happened”, and to a certain extent, “Why did that happen?”. I do think, as a slight caveat, in annual progress reports, you usually don’t have quite as much space to expand on the “why” as you do within the evaluation report, and I think that’s why they work together, and I’ll get to that more in a minute, but there is an element of that within the annual report as well. And then what the annual progress reports are asking about is “What are you going to do next? What are your next steps?”.
So, what does that mean in terms of trying to understand the description and why they’re asking those pieces of information? In terms of “what did you say you were going to do?” is when you’re providing an overview of the project, including the goals that you were planning to achieve during that reporting period. It’s really helpful for the program officer and for the funder to have that information right in front of them when they start to review the next part, which is “What happened?”. This is where you’re going to describe what you were able to accomplish — depending on the size of the grant, you may have to provide additional detail around that. That may come in the format of including what challenges you had if there were personnel changes, and it can also relate to budget information too because how you used your budget also is an indication of what happened. The finally, providing out some information in regard to what your next steps are going to be – what is that next year, or whatever that reporting cycle is going to entail? To show you what that looks like in practice, I’m going to pull out some annual reports from three different types of funders.
The first funder is actually from a private foundation, and in this context, they didn’t really have a very formalized reporting template, so the questions word just in a Word formation, and these are some of the questions they asked: they asked to describe activities during the reporting period, the progress made accomplishments, describe the proposed work, describe progress made toward accomplishments, any setbacks, and then what you’re going to do for the next reporting period. So, when you think about those three basic elements, you can see how that first question is really tapping into the “what happened” component. The second element, then, is really tapping into “what did you say you’re going to do?”. I really like this example in particular, in terms of really understanding conceptually what funders are asking for in your annual report, because I think my inclination would probably have been to ask “describe the proposed work first because it highlights “what did you say you’re going to do” and then what you actually did follows. So in that reverse order, if you weren’t necessarily applying that conceptual frame, it may be harder to really think about what they’re asking for or what you want to be able to provide to the funder in that response, but an understanding that they’re really just asking about what you said you were going to do and what did you actually do, these basic elements will help you to be able to respond to that annual report. And as you can see, the final questions really tap into that question of “What are you going to do next?”. In a similar vein, that’s the same way in terms of looking at the Department of Education’s grant performance reports. Now, they do have a much more formalized process so let me give the cover page that you would have in terms of your grant performance.
So in this situation, the first part is asking about what you said you’re going to do – the information about the project itself that’s provided. Then it’s asking about budgetary information in terms of “what happened”. Then there’s also some information at the bottom too, and again, some of this doesn’t completely map onto the questions, but in a broad way, you can think about that final question as asking about what you’re going to do. When you actually get to the grant performance information it’s again asking about “what did you do?”. In the next portions, there’s a question asking about an explanation of progress. In this portion, you can talk some about what you did, but you should also add in what you’re planning to do as well. While that question doesn’t necessarily spark that that’s what they’re asking for, the instruction sheet that accompanies this is what that information was provided. Now looking at the National Science Foundation’s template, as with the Department of Education, this is just giving a sample of some of the questions they’re asking and not the complete template that’s provided. So, in the NSF template, they’re asking about what are the major goals, or what were we trying to do? Then they are also asking about what you’re accomplishing and that theme of what you’re going to do next. So, you can see how across these three different funders, the basic information falls in those three broad categories. The take-home lesson from that is that annual reports can be distilled into conceptual components and by distilling it into conceptual components, completing the annual report becomes just a little bit easier.
So, let’s go on and let’s start talking about best practices. In thinking about various best practices I will be sharing out with you, it’s just important to keep in mind that these tips are gleaned in part from my own experience in working with a number of project teams, but more importantly, it’s also some direct conversations that I’ve had with program officers, principal investigators, as well as other evaluators. Some of these are kind of broader themes of categories to keep in mind as either you’re writing the annual report, or in terms of the content for the report. It’s not an exhaustive list, as I’m alluding to, but it really does tap into some of those broader themes that have emerged over time.
So the first and the best, best practice is to make sure to be concise. When I’m thinking about being concise, it really prompts one of my favorite quotations, “Because I didn’t have time to write a short letter, I wrote a long one.” One of the reasons why I love this quotation is because it really speaks to the idea that it takes a lot of work and a lot of effort to be able to be concise and succinct, and that’s not necessarily an easy place to get to. I had an experience when we were working on evaluation reports, and we put in a lot of effort, and it yields a relatively short document. I think we worked really hard on this, but I think what we’ve really done is made it very concise and sucking. To able to do that leads to the next best practice – make sure to start the process early.
It’s really important that you give yourself enough time to be able to be succinct and concise. To give a frame for how to do this, I’m going to reference a whitepaper that was written by Tara Sheffer and colleagues at Columbus State Community College (Columbus State). And over the last 10 years, Columbus State has become what I would say is a grant powerhouse – they have a number of NSF grants, a number of Department of Labor grants, and state grants. In really being able to implement those larger number of grants, they’ve formalized their annual report writing process. This whitepaper is available on our website as well, but I want to briefly highlight some of the lessons from this whitepaper. In this paper, they argue that there are about five steps within the annual report writing process. Step one is in regard to preparation. You want to make sure that you’re fathering information out – what data do you have available that you want to be able to share in your annual report. What’s also really key (and Columbus State isn’t the only entity that does this – I’ve seen this with other entities in other colleges as well) is to make sure that everybody who is a key member on that project gets in the room together to be able to brainstorm around the annual report. I’ve seen instances where people have been writing more so isolation than writing in the group. It seems as though being able to brainstorm in a group is helpful in terms of generating various ideas and catalyzing the writing process. The next step is writing and editing. Once you get your ideas down on paper, someone will be tasked with translating that into much more formal language and that will get kicked band and forth to make sure it’s ready for submission – the next phase. What’s important to keep in mind is depending on your program officer and the program you’re responding to, the program office may give you a very quick response of may have some additional questions for you, so you want to make sure you give enough time to allow for that Q & A and that review it before it’s actually approved. That approval piece is really important, particularly in this NSF grant because if you have an overdue annual report, then you may not get your additional funds for the next year, it could impact on submitting additional grants, and it may also impact what’s occurring with other grants that you may be working on. So, it’s important to note that just because it’s been submitted, doesn’t mean it’s necessarily been approved. In thinking about that, let me provide a suggested timeline. Preparation begins about 14 weeks before the actual due date. You’ll want to give about four to five weeks for writing and editing. Then you have your submission that will give time for the program officer to review if that’s needed, and then it will be accepted. So that’s just something to think about in being able to start early.
Another best practice is just in regards to making sure to share out challenges. I know it’s really difficult to share out the obstacles that you’re having, but I think it’s really beneficial to do that and that’s something I’ve heard from PI’s and some program officers as well. One frame to think about why you may even have those challenges is nested in the way I like to compare research and evaluation. In research, you have your hypothesis, and you develop a research question, test it, analyze the findings, and refine the hypothesis. In a similar way, that happens with evaluation. What’s key in thinking about this comparison between research and evaluation as it relates to annual report writing, is that the implementation is where you’re testing your theory and your hypothesis. So, it’s not uncommon when you’re implementing a project that you are going to have to make tweaks and run into roadblocks. You want to make sure that you’re sharing out your challenges so that the program officer has a sense of what the story of the project is, and more importantly, will be able to provide suggestions on how to move beyond some of the roadblocks you’re experiencing. Those are the first three of six different practices I want to share.
Let me go ahead and move on to the next portion with the additional best practices. Something to keep in mind is the difference between the annual report and the evaluation report. The annual report has more required elements that you have to include in the document where the evaluation reporting is slightly less structured. This is beneficial because it enables you to tell more about the story and more about the background of what the project is and what’s occurring. Another difference is that the annual report is more limited in the time frame and the evaluation report can tell the entire life of the project. Where the annual report may only tap into a 12-month time frame, the evaluation report can talk about multiple years of that grant. Another piece is that the annual report tends to be slightly more results-focused – that is not to say that the evaluation report is not results-focused, but I think the evaluation reported is also “why” oriented. So, these two documents together are really important in terms of telling the story, and the evaluation report can help you to inform your annual report. So, I know the questions asked was, “What happens if you’re not given the evaluation report?” and there are a couple of things to keep in mind. One, you know that during the first year, sometimes the timing between the evaluation report and annual report may be difficult to ensure everything is syncing. But even if you’re not really given a formal evaluation report, hopefully, there’s data you can obtain. And hopefully, when you’re doing that brainstorming component as part of the preparation and baselines of report writing, you invite the evaluator in so they can share out evaluation findings. So even though you don’t have the formal evaluation report, you should still be able to access that information and use it to inform the annual report.
Another best practice is regarding completing the annual report, and this is specific to the National Science Foundation, but you want to also refer to the certification of effort that is completed. Where this is coming from is in the annual report for NSF, there’s a portion that’s asking for how to calculate nearest person month. We provide this information as a resource because we get a lot of questions about how to make that calculation – in fact, it’s the number one search topic with our blogs. The nearest person month that is reported in your annual report also relates back to the certification of effort. Certification effort is probably mentioned in multiple places within NSF documents, but it is mentioned in the prospect for a new reward where on a periodic basis (sometimes it’s twice a year, sometimes it’s every quarter), and you’ll want to be sure those are syncing up and referring back to your certification effort as you’re actually completing out your nearest person month.
The next, and final, best practice is to remember to remove student-level information. You have this in a report, and in that annual report, you may want to share out some specific information related to what a student did and what types of scholarships they received, or what have you, so you’ll need to remove that. Keep in mind that the reason for removing this information is that this is potentially a public document, and someone could do a freedom of information request and obtain your annual report. So again, make sure you’re not including personal student information or any personal information overall. So essential, the take-home lesion for that is if you used a structured process incorporating some best practices, the annual report will be a less stressful undertaking.
Well, we are getting close to time here, so let me just go over a couple more items quickly. One I did want to highlight another resource that will be available on the website, and you receive this in the follow-up email when you receive the recording for the webinar. I received from a PI who is allowing me to share this, ad annotated version of the NSF annual report template. It has additional comments that help you in terms of completing the NSF annual report. Also, I want to let you know that on June 17th, we are going to have another Coffee Break Webinar on Demystifying Writing an Evaluation Plan, so make plans to participate in that.
That’s it! Thank you so much for coming today. If you have any questions, please don’t hesitate to reach out, and if we have any questions we didn’t get to, we’ll make sure to follow up with you as well. Enjoy the rest of your day.
