We graded writing and organizational compliance independently, with a separate set of judges for each. For more on the judges, see “Who Did the Grading?”
For the organizational compliance: we visited each agency’s Plain Language page (sometimes titled “Plain Writing” page); this page is required by the Plain Writing Act, so having an updated page was one of the criteria. Other criteria were staffing, which included having available to the public the names and contact information for the two plain language positions required by law; annual training that complies with the 2011 OMB guidance for the Plain Writing Act; and annual reports required by the law.
For the writing evaluation: in late July, the Center notified the Plain Language contacts from each agency what pages we would be grading. We asked these contacts to provide information on audiences, purposes and any research they had done on the pages to be graded. All information provided by the agency teams was then included in the evaluation material provided to judges. In late September, lead judge David Lipscomb gave our 15 judges their assignments, with three judges reviewing and providing written comments on each page. To ensure a consistent approach, all judges completed a “norming exercise” on sample webpages at the beginning of grading, and judges had frequent check-ins with team leads and the overall Report Card lead, David Lipscomb. Once grading was completed at the beginning of November, we averaged each agency’s urgent help page grade and most-visited webpage grade to arrive at their final writing grade.
2019 Report Card Grading Criteria
Understanding audience needs
- Is it clear what the audiences should learn or do using this entry?
- Are the writing, tone and presentation appropriate for the audiences?
- Did you get the sense that the writers genuinely want the audiences who use this product to succeed?
Style or voice
- Do the writers follow plain writing principles? For example, do writers use relatively short sentences and active voice, while avoiding noun-strings, hidden verbs (nominalizations), and delayed verbs?
- Does the voice seem sincere and credible?
- Do the tone, choice of words and conversational style convey respect for the target audience? For example, do the writers use reader-oriented language, including “you” and “your”?
Structure and Content
- Are the sections clearly organized and labeled?
- Will the labels help audiences predict what is in each section?
- Do the writers create effective transitions between sentences, paragraphs and sections?
- Is the content presented in an order that tells a story or helps audiences complete a task?
- Do they convey key content while effectively winnowing unnecessary details?
- Do the writers provide relevant information in a balanced way, without overselling or underselling their points?
Information Design and Navigation
- Do the typography, color, and white-space grab and guide the audiences’ attention?
- Do the layout and presentation make the product easy to scan?
- Can you tell by glancing where the important information or action is?
- Does the navigation offer a visible, guided path through the content?
Pictures, Graphics and Charts
- Do the pictures, graphics, or charts support the content?
- Will audiences understand the point of the chart or graph?
- Do the visuals help audiences understand important points better or guide them on how to take important steps?
- Conversely, are the images and graphics included merely as decoration? Or would this page be easier to understand if the writers had chosen more or different graphics?
- Will the target audiences be able to find, understand and act confidently using what they learn in this entry?
- Will the product help the authoring organization achieve its business goals (for example, increased customer self-service, enhanced mission, better consumer decisions)?
- Would you use this product as an example of effective plain writing and information design?