By Kathaleen Reid-Martinez, PHD
Have your every found yourself asking questions such as where are we on this project? Where should we be? What is the best thing to do? Or maybe even asking, How do I even find out what is best to do? Or, after you finish a hard project and you’re tired but feeling proud of what’s been done, have you ever had those niggling questions at the back of your mind–questions like did we really do a good job, or do we just think we did? Did we really say what we wanted to say, or have we heard it so many times until we know what we are talking about–but does anyone else?
If you are like I am, these kinds of questions plague each step of my work. To find answers to such questions, we know that we must turn to research, but what kind?
I suggest that we look at each of our projects using what’s known as formative research.
Formative research is research during the development and production of a program aimed at improving the program’s effectiveness. It helps us evaluate our project from its beginning to its end. Special emphasis is placed on collecting data and perspectives to improve our messages during the course of creation. It is a step-by-step process combining different research methods to identify, construct, implement, and evaluate a program based upon our predetermined goals that meet the needs of our target audience.
Throughout this evaluation process, you may use both qualitative and quantitative research methods to ensure that the best answers possible to your questions regarding audience, content, and effectiveness of the programs you are designing. Formative research may be done simply and easily or it may be done with more complexity–it all depends upon your time, energy, expertise, and budget. Whichever route you go, remember, this evaluative process is essential to the success of your projects, while it is easily adapted to the strengths and the limitations of your own unique situation.
The purpose of formative evaluation research is to assure that our messages are appropriately tailored to our audience. Proper research will allow us to develop new solutions to communication problems in our working environment.
So let’s take a look at some characteristics of formative research to see how it can fit in your everyday working world and can answer some of the above questions:
Characteristic #1: Formative research is practical and directly relevant to an actual situation in our working world. This is not an isolated laboratory experiment, but a practical way of dealing with your projects in your daily work environment.
Characteristic #2: It provides an orderly framework for problem-solving and new developments that is superior to just prejudices, impressions, and feelings about how we should present our messages. This process of research can confirm or disprove our hunches and intuition about our work.
Characteristic #3: Formative research is always flexible and adaptive, allowing for changes. With formative research, we exchange complete control over the research process for responsiveness and on-the-spot experimentation and innovation.
Characteristic #4: This research attempts to be systematic, but because it is flexible and adapts to problems in the environment, it cannot contain scientific rigor.
Thus, the findings are not as strong as those from more stringent forms of research. However, having said this, formative research must not underestimated. It can provide powerful information—information that will enhance our work and make our messages more effective.
Examples of the types of projects that can use formative research might include recruiting staff for specific tasks, developing partnerships with indigenous organizations, and designing media programs for broadcast.
Let’s take a look at one of these. When we are designing a video or television program, we often begin by asking basic questions. For instance, Why are we designing this project? What’s our message? Who are we trying to reach? That is, who is the target audience and what do we need to say to them? Is the broadcast program really the best medium for reaching this audience? If the answer is yes, we need an electronic broadcast program, we then ask such questions as what should be the style and content of the program? What should be its length? And after the broadcast, we might ask, Were we successful? Did we really reach the people we wanted to? Did they understand what we were saying? And finally, we might want to know, what questions does the audience still have? Can we answer those in another program?
To find the answers to these and other questions we turn to our research. Research is often perceived as something “out there” and not directly related to what you have to do “here and now.” This discussion should help you see where formative research is not “out there” but is essential for your work now as you are developing it.
Now you say, this is fine, but how do I go about this? Well, let’s begin with the basics. First you must define the problem. What issue needs to be addressed? What is it that needs to be changed? What relationship does your organization have to this problem or issue? From this you can establish you purpose and your goals.
Simply put, your purpose is a clear, succinct statement of the reason for your media project. This should address the nature of the problem and the desires of your organization in light of the needs of your audience. Failure to correctly identify the problem can result in a loss of time and resources for a project that misses the mark. Interviews with potential audience members as well as others within your organization or related experts are helpful at this point. Questions to ask that will help you determine the purpose and goals of your project could include:
How does this fit within the mission of your organization?
What is the problem? Be specific in your answer. Probe for details that you might miss as you first develop your ideas about this project.
Who should receive the message you will create? Be specific in your answer. What are the potential ramifications of your message? Do the potential positive effects outweigh the potential negatives?
After defining your purpose and goals, your research will wrestle with only three other components–your objectives, means, and measures. In a nutshell, your formative research will evaluate each of these areas to ascertain what your goals and objectives should be and whether or not your message and its medium is developed adequately to achieve them.
STAGE 1: PROGRAM PLANNING (PRE-PRODUCTION RESEARCH)
Stage One involves the planning and initial research associated with beginning the production process. Sometimes this is referred to as Pre-Production Research. This includes four primary areas: researching the audience, identifying the needs, reviewing other sources of information, and establishing your objectives.
I. Identify Your Objectives
The fourth area of planning is to identify your program’s objectives based upon your research findings which you have just collected. Make certain that your objectives are well-written. The general objective should contain a specific concept which you desire to teach or persuade the audience to follow. Your specific objectives should clearly identify what specifically you are seeking from the audience members who view your message. Remember, don’t let the specific objective be vague, but state clearly what you desire from them. Do you expect them to grasp the concept of salvation? Of compassion? If so, after they view your program, they should be able to discuss the role of Christ in salvation, to identify their own spiritual condition, to contrast their concept of compassion with the one presented in your program, State the objectives in clear, specific, pragmatic language so they can be clearly measured. (See Objectives discussed above. )
- Review the Literature and Consult Experts
Know what they and experts have done and written. Check related books, magazines, and other printed materials. Learn whether others have met similar problems or achieved related objectives. You should be checking the content of the program, the best ways to produce your information, and for any limitations of your medium in your particular culture. Consulting with experts while developing your program can enhance your work throughout the process.
- Research the Target Audience
Questions you might seek to answer could include: What are the audience characteristics? What can you find out regarding their demographics, their family structure, religious beliefs, and any other attitudes relating to your subject? What about any government restrictions or policies that may need to shape your work? What cultural ideas or events may color your audience’s interpretation of your message? Both quantitative and qualitative research can be used to help you ascertain this knowledge. For example, you could conduct focus groups or in-depth interview with key people in the community. Don’t forget to use any printed resources such as those found in libraries and even on the Internet.
- Identify Audience Needs and Select a Medium
Clearly state the problem or need of the audience based upon your research. A need is simply the difference between what is and what ought to be. A basic comparison of these two should result in a basic assessment and clear statement of the audience’s needs. Prioritize these needs. Then use them to set program goals and objectives.
While collecting your information in the previous step, ask specific questions to help you ascertain audience needs. These could include:
What kind of information is already available?
What kinds of information should be produced?
What information is needed?
How will this information be used?
How is it best disseminated?
Is an electronic broadcast program the best means of presenting this message?
Does this information exist elsewhere? If so, producing it again is a waste of time and money.
Ascertain mass media preferences of your audience. This should be based on media available. For example, if television or radio is available, determined how much time they spend watching television or listening to the radio and what type of media content they prefer (i.e., news, dramas, documentaries, etc.)
STAGE 2: PROGRAM PRODUCTION
Program production involves what is sometimes referred to as the “pre-release’ research. This Stage of formative evaluation involves using the above information to write your script and to test it prior to release to your audience. You can use a number of research methods to gain your information. These include:
Focus groups
Individual in-depth interviews
Central-location intercept interview
Self-administered questionnaires
Theater testing
Day-after recall
I. Writing the Script
Good production and artistic presentation are important parts of your work, but the heart of your production is the script. The final test that will determine the success of your program will be simply, was your message understood? A good script that is contextualized to each specific culture is essential. Contextualization requires more than simply good language translation from culture to culture. It entails characterizations, story development, and settings that are understandable to each culture. Program length must also be determined and factored in as part of the design of the script.
2. Testing the Script
After writing the script, it’s time to try out the program before it is actually produced. This can prevent the waste of much time and money for programs that are not used or that fail. Pre-testing can eliminate weaker approaches and identify the most promising ways to develop your concepts in your program. Conceptualization, credibility, message themes, persuasive arguments, stylistic devises, and contextualization can all be measured using focus groups and in-depth interviews. Once these elements with the most potential are selected, complete messages can be created in rough form to pretest. This may be done using live or recorded script readings, storyboards, animators, drama teams, and informally shot action scenes. Research using these forms of presentation should reveal:
Audience attention to the program topic and style of presentation.
Audience comprehension of the message.
Identification of strengths and weaknesses of the program.
Personal relevance.
Gauge audience response to determine sensitive or controversial elements.
3. Producing, Testing and Revising the Program
During production, the program’s specific purpose should be reviewed throughout the process so that new artistic opportunities do not destroy the program’s effectiveness. Note the ways in which deviations from the script alters the final product. After the program is produced, it must be tried out on a larger population.
This testing is often referred to as field testing. The program is now presented to a ‘typical” audience in a “typical” setting. Quantitative research is Often helpful at this stage. Be careful to maintain an appropriate audience representation for your sample population so that your research can be more accurate.
When the results of the field test have been analyzed, make any final revisions, if needed, to your program. Be certain that you do not let your own bias interfere with your assessment of any negative results from your research. Step back from your own preferences so that your program achieves your stated objectives.
STAGE 3: POST RELEASE RESEARCH
After the program has been in distributed for a period of time, you must evaluate its continued relevance to the audiences in various settings and contexts. This will allow you to measure immediate and delayed effects Of the program. You can track the dissemination process and assess approximate number of viewers. Responses to the program can be determined through personal follow-up interviews immediately after the showing of the program, through requests for additional information, and through dayafter recall interviews. Where appropriate, quantitative research using survey and experimental designs could be used to further ascertain manifest and latent consequences of the program.
Most importantly, information gleaned from these three stages can point to your strengths and weaknesses so that quality of future programs is enhanced and made even more effective. It will help to determine the goals and objectives for the next project and ensure continuity in definitions and concepts developed in each series of programs.
CONCLUSION:
Formative Evaluation is most commonly used within the context of a long term study of the interaction of numerous media, such as film, radio, and print media. The concepts of purpose, goals, objectives, means and measures along with systematic research can be used to guide a long-term mediated campaign.
–
