|
|
|
Please press CTRL+P to print this page
2013 Model Practices (Public)Application Name: 2013 Model Practices (Public) : Salt Lake County Health Department : Restaurant Inspection Website Tool for Reducing Critical Violations Applicant Name: Mr. Gary L. Edwards, MS Application Title: Restaurant Inspection Website Tool for Reducing Critical Violations Please enter email addresses you would like your confirmation to be sent to. bwaters@slco.org Practice Title Restaurant Inspection Website Tool for Reducing Critical Violations Submitting LHD/Agency/Organization Salt Lake Valley Health Department Head of LHD/Agency/Organization Gary Edwards Street Address 2001 South State Street S2500 Practice Contact Person Royal DeLegge, PhD Title Director, Environmental Health Submitting LHD/Agency/Organization Web Address (if applicable) http://www.slvhealth.org/ Provide a brief summary of the practice in this section. This overview will be used to introduce the model or promising practice in the Model Practices Database. Although this section is not judged, the judges use it to get an overall idea about your practice. You must include answers to the following questions in your response:
• Size of population in your health department’s jurisdiction
• Who is your target population/audience, what is the size of your intended population/audience for this practice and what percent of your target population did you reach?
•Provide the demographics of your target population (i.e. age, gender, race/ethnicity, socio-economic status)
• Describe the nature and gravity of the public health issue addressed
• List the goal’s and objective(s) of the practice and clearly link them to the problem or issue the practice is addressing.
• Describe the potential public health impact of the practice, and the likely effectiveness of the practice being implemented as intended, and the ease of adoption of the practice by other LHDs.
In your description, also address the following
• When (month and year) the practice was implemented.
• Briefly describe how the practice was implemented, what were major activities, and any start-up and in-kind costs and funding services.
• Outcomes of practice (list process milestones and intended/actual outcomes and impacts.
• Were all of the objectives met?
• What specific factors led to the success of this practice? Salt Lake Valley Health Department SLVHD provides services to 1,029,655 persons living within Salt Lake County, Utah. Salt Lake County consists of 16 incorporated cities and an unincorporated area comprised of various townships and communities. Salt Lake County is a culturally diverse community with notable representation from many different ethnic groups. The median age is 28.5 years.
Approximately 3,924 permitted food establishments in the County are inspected by personnel within the Bureau of Food Protection. Over 5,000 routine restaurant inspections are conducted annually in Salt Lake County and posted on the Department’s website which provides information regarding restaurant inspection results, scores, rankings, and restaurant closures.
In April 2009, SLVHD launched the food inspection website targeting residents who eat at the County permitted restaurants and provides general public access to restaurant inspection data. The website was designed by a team in the Bureau of Food Protection who then collaborated with the Utah Restaurant Association, Utah Food Industry Association, and the Salt Lake Environmental Quality Advisory Commission for feedback.
The objectives of the website are:
(1) Improve public health. Foodborne diseases result in 9.4 million illnesses, 55,961 hospitalizations and 3,151 deaths annually in the United States. One in six Americans suffer from foodborne illness each year. A primary objective of the website is to reduce critical violations and protecting public health. Effectively reducing the number of critical violations cited during routine restaurant inspections through internet posting of inspection results should decrease the occurrence of foodborne illnesses, thus improving public health.
(2) Benefits the public. The inspection website offers meaningful information to the general public. It contains a multi-field search engine in which a user is able to check a restaurant’s inspection score and rating, view each violation found by inspectors, the public health rationale for each violation, the actual food code verbiage for each violation, and whether or not each violation was corrected on site. Other useful features include an inspector jargon glossary, a top ten list of most common violations in Salt Lake County, a list of current food establishment closures, links to FDA and CDC foodborne illness web pages, and a comparison function that allows a user to compare up to three restaurants side by side to see how they rate.
(3) Benefits the food service operators. Many of the web site features available to the public also benefit food service operators. Establishment owners are able to keep track of how their establishment(s) performed on the most recent inspection, as well as on inspections dating back to January 2008. Since rankings are relative to other food establishments within a risk level, owners and managers can see how they match up to the competition. Managers have all of the food code and public health rationale information literally at their fingertips to aid them in understanding why a violation was cited, and in training their employees to avoid practices that lead to violations. Critical violations and critical factors appear in red to highlight the most important items on which food mangers should focus.
(4) Benefits SLVHD. Food service managers’ interest in their inspection results noticeably increased after the launch of the inspection web site. Overflow: Please finish the response to the question above by using this text area. Please be mindful of the word limits. Not only were their inspection scores on display, but every detail of all of their violations was readily available to everyone in the general public. Shortly after the web site went live, corporate executives of two major restaurant chains requested meetings with SLVHD so that they could become better informed about how to perform well on routine inspections.
You may provide no more than two supplement materials to support your application. These may include but are not limited to graphs, images, photos, newspaper articles, etc. Describe the public health issue that this practice addresses. (350 word limit) Preventing foodborne illness is an important public health issue. Foodborne diseases result in 9.4 million illnesses, 55,961 hospitalizations and 3,151 deaths annually in the United States. The annual economic cost of foodborne illness is estimated to be $152 billion. In 2008 the projected incidence rate for all reported foodborne illnesses in the United States was 40.22 per 100,000 population. The estimated incidence in Salt Lake County for the same time period was 36.67 per 100,000.
Most foodborne illnesses can be linked to improper food handling practices. State and local health departments are responsible for regulating food safety in restaurants and implementing those new and innovative strategies. Health departments have relied primarily on training and inspections to ensure proper food handling behavior. These two procedures have not proven effective in creating long term behavioral compliance. Studies have shown, however, that improving food safety practices, resulting in better inspection scores, reduces risk and improves public health. In response, the United States Food and Drug Administration (FDA) has called for more innovative and effective strategies to improve food safety practices in retail food establishments. Foodborne disease is a winnable battle identified by the CDC. Posting inspections scores on the internet is an innovative and effective strategy for reducing critical violations identified during routine inspections.
What process was used to determine the relevancy of the public health issue to the community? (350 word limit) The relevance of foodborne illness and effective strategies for reducing critical violations in retail establishments was verified through a literature review of studies and information gathered from the U.S. Food and Drug Administration (FDA) and the Centers for Disease Control and Prevention (CDC) which suggest an association between inspection scores and foodborne illness. Studies conducted in Seattle-King County, Washington, and Los Angeles County, California concluded that inspection scores can identify restaurants with increased risk of foodborne illness outbreaks. A study in California compared restaurants associated with foodborne illness outbreaks with previous inspections and found inspection scores to be predictive of foodborne illnesses. How does the practice address the issue? (350 word limit) The heightened interest in inspection scores has ultimately led to cleaner kitchens and reduced numbers of violations. It has prompted food service managers to take a more active role in training their staff to control critical factors. A comparison of average inspection scores from 6 months before the web site launch and 6 months after the web site launch revealed an average decrease of 4.3 points per inspection. The implementation of a web-based system for public access to restaurant inspection results appears to have had an impact on the frequency of critical violations noted by inspectors. Based on these results, implementing such a web-based system may lead to a 20 to 30 % reduction in the number of inspections with critical violations. Posting inspection scores on the internet appears to be an effective and efficient tool for protecting public health. Please list any evidence based strategies used in developing this practice. (Provide links or other materials for support) The practice of posting restaurant inspections on the internet was a result of a review of studies conducted by the U.S. Food and Drug Administration (FDA) and the Centers for Disease Control and Prevention (CDC) which suggest an association between inspection scores and foodborne illness. Additional studies conducted in California concluded that posting inspection scores in restaurant windows appear to have an effect on the frequency of foodborne illnesses. Internet postings were the logical next step. Is the practice new to the field of public health? If so, answer the following questions. Yes What process was used to determine that the practice is new to the field of public health? Please provide any supporting evidence you may have, e.g. literature review. Although posting inspection scores in establishment windows has been common in local jurisdictions, internet postings are relatively new. Analyzing the website’s effectiveness in reducing critical violations identified during routine inspections is unique to SLVHD. Municipalities nationwide have begun posting inspection scores on the internet to improve the accessibility of scores to the public; however the efficacy of internet postings is unknown. A search of PubMed and EBSCOhost databases yielded no papers examining the impact of internet access to inspection scores on subsequent scores or foodborne illness outbreaks.
How does this practice differ from other approaches used to address the public health issue? Internet posting of inspection scores differs from window posting in several key areas. Because the inspection results are not posted immediately, the inspector is not pressured to give the restaurant a favorable score, thus the score is a better representation of the restaurant. It is impossible for the restaurant owner to alter the inspection results once posted on the internet.
Is the practice a creative use of an existing tool or practice? If so, answer the following questions. No What process was used to determine that the practice is a creative use of an existing tool or practice? Please provide any supporting evidence you may have, for example, literature review. What tool or practice (e.g., APC development tool, The Guide to Community Preventive Services, HP 2020, MAPP, PACE EH, etc.); did you use in a creative way to create your practice? (if applicable) (300 word limit total)
a. Is it in NACCHO’s Toolbox; (if not, have you uploaded it in the Toolbox)?
b. If you used a tool or practice to implement your practice, how was your approach to implementing the tool unique and innovative for your target area/population? How does this practice differ from other approaches used to address the public health issue? Who were the primary stakeholders in the practice? Salt Lake Valley Health Department Bureau of Food Protection, Utah Restaurant Association, University of Utah Division of Public Health, Salt Lake Environmental Quality Advisory Commission What is the LHD's role in this practice? The website was researched and designed by a team in the Bureau of Food Protection. Bureau personnel monitor the website for accuracy and completeness. What is the role of stakeholders/partners in the planning and implementation of the practice? The Utah Restaurant Association, Utah Food Industry Association, and the Salt Lake Environmental Quality Advisory Commission provide input and feedback to the Bureau. The effectiveness analysis was a collaboration between SLVHD and the University of Utah. SLVHD personnel provided the data, conducted literature reviews, developed the analysis models, conducted the analysis, and co-authored the report. University of Utah faculty provided direction and advice regarding measures, models and processes, and co-authored the report. What does the LHD do to foster collaboration with community shareholders?
Describe the relationship(s) and how it furthers the practice's goals. SLVHD has an ongoing relationship with several of the departments at the University of Utah, Westminster College, and Salt Lake Community College in addition to the Utah Restaurant Association and the Salt Lake Environmental Quality Advisory Commission. These relationships have been reciprocal over the years wherein both the shareholders and SLVHD have initiated and benefited from the various arrangements. SLVHD has benefitted from their expertise and the community stakeholders have gained access to data and other SLVHD resources.
Communication and collaboration with the community are part of the SLVHD values. Many of the programs administered by SLVHD are collaborative efforts with community stakeholders. Program success indicators include 1) programs which are based on community needs and 2) the community is knowledgeable and satisfied with our services.
The quality, credibility and results of the evaluations conducted in this study were greatly enhanced due to the collaboration with the University of Utah faculty. Their expertise in designing and conducting effectiveness evaluations saved SLVHD countless hours of time and resources. The study was reviewed and critiqued throughout the process to ensure an accurate and valuable assessment.
Describe lessons learned and barriers to developing collaborations. Stakeholders need to be involved in every step of the program. Resistance from some stakeholders regarding the most appropriate method of ranking the restaurants slowed the process at critical times. It is advantageous to have those who will be conducting the analysis participate in the process to ensure all of the required data will be collected. Stakeholders need to remember the purpose of the website and not be distracted by their personal agendas or preferences. Evaluation assesses the value of the practice and the potential worth it has to other LHDs and the populations they serve. It is also an effective means to assess the credibility of the practice. Evaluation helps public health practice maintain standards and improves practice.
Two types of evaluation are process and outcome. Process evaluation assesses the effectiveness of the steps taken to achieve the desired practice outcomes. Outcome evaluation summarizes the results of the practice efforts. Results may be long-term, such as an improvement in health status, or short-term, such as an improvement in knowledge/awareness, a policy change, an increase in numbers reached, etc. Results may be quantitative (empirical data such as percentages or numerical counts) and/or qualitative (e.g., focus group results, in-depth interviews, or anecdotal evidence).
List up to three primary objectives for the practice. For each objective, provide the following information: (750 word limit per objective)
• Performance measures used to evaluate the practice: List the performance measures used in your evaluation. Depending on the type of evaluation conducted, these might be measures of processes (e.g., number of meetings held, number of partners contacted), program outputs (e.g., number of clients served, number of informational flyers distributed), or program outcomes (e.g., policy change, change in knowledge or attitude, change in a health indicator)
• Data: List secondary and primary data sources used for the evaluation. Describe what primary data, if any were collected for each performance measure, who collected them, and how.
• Evaluation results: Summarize what the LHD learned from the process and/or outcome evaluation. To what extent did the LHD successfully implement the activities that supported that objective? To what extent was the objective achieved?
• Feedback: List who received the evaluation results, what lessons were learned, and what modifications, if any, were made to the practice as a result of the data findings.
• Performance measures used to evaluate the practice: List the performance measures used in your evaluation. Depending on the type of evaluation conducted, these might be measures of processes (e.g., number of meetings held, number of partners contacted), program outputs (e.g., number of clients served, number of informational flyers distributed), or program outcomes (e.g., policy change, change in knowledge or attitude, change in a health indicator)
• Data: List secondary and primary data sources used for the evaluation. Describe what primary data, if any were collected for each performance measure, who collected them, and how.
• Evaluation results: Summarize what the LHD learned from the process and/or outcome evaluation. To what extent did the LHD successfully implement the activities that supported that objective? To what extent was the objective achieved?
• Feedback: List who received the evaluation results, what lessons were learned, and what modifications, if any, were made to the practice as a result of the data findings.
Website Objective.
The primary objective of the website is to reduce critical violations by providing restaurant inspection information to the public and restaurant owners or operators. The design of the website included a search engine in which a user is able to check a restaurant’s inspection score and rating, view each violation found by inspectors, the public health rationale for each violation, the actual food code verbiage for each violation, and whether or not each violation was corrected on site. Other useful features include an inspector jargon glossary, a top ten list of most common violations in Salt Lake County, a list of current food establishment closures, links to FDA and CDC foodborne illness web pages, and a comparison function that allows a user to compare up to three restaurants side by side to see how they rate.
Performance Measures. The occurrence of five specific, FDA defined, critical violations were included as outcome measures: poor hygiene practices (lack of handwashing and eating, drinking or using tobacco in the food preparation area), improper holding temperatures (hot and cold holding temperature citations), inadequate equipment cleanliness (dirty equipment, utensils or food contact surfaces), lack of protection from cross-contamination (food separation, packaging and segregation citations), and improper sanitizer concentration (improper manual and mechanical ware washing chemical sanitizer levels). These are referred to as ‘target critical violations’. These violations were selected because they were included in the domains identified by the FDA as “most commonly reported to the Centers for Disease Control and Prevention (CDC) as contributing factors in foodborne illness outbreaks”.
Data. This study uses a longitudinal analysis of inspection results. Study data were limited to routine inspections performed by licensed environmental health specialists at full service and fast food restaurants in Salt Lake County, Utah. Inspections conducted 12 months before and after the website launch were included. Only those restaurants which were inspected at least twice before and twice after launch were included in the study. A total of 2,995 inspections conducted at 795 restaurants were evaluated. Secondary data were obtained from the SLVHD database and redacted to ensure anonymity. Univariate and bivariate methods were used to assess the changes.
Overflow (Objective 1): Please finish the response to the question above by using this text area. Please be mindful of the word limits.
Multivariable methods were used to assess the changes in inspection results. Logistic regressions using a Generalized Estimating Equation (GEE) were used to account for the repeated measures from each restaurant. The outcome variables were each dichotomous indicating whether the restaurant was cited for that type of critical violation. A first order autoregressive (AR1) correlation structure was assumed to account for any correlation between the previous and current inspection. Based on the quasi-likelihood criteria, the AR1 structure performed as well as other correlation structures. Odds ratios were used to describe the effect of the intervention. Data were analyzed using STATA software version 9 (College Station, Texas).
Models were adjusted for restaurant type (full service compared to fast food), inspector experience and season. Experienced inspectors were defined as those who conducted 200 or more inspections during the study period. Season was grouped into four 3 month periods: Summer (June, July, August), Fall (September, October, November), Winter (December, January, February), Spring (March, April, May).
Evaluation Results. The proportion of inspections with any of the targeted violations was lower during the twelve months after the launch compared to before the launch. The difference in proportions ranged from 2.07 (cross-contamination) to 8.99 (equipment cleanliness).
The probability of having a violation decreased in routine inspections conducted after the website launch when adjusted for inspector experience, risk level and seasonality. The adjusted odds ratios, ranging from 0.64 to 0.80, were statistically significant for all critical violations, with the exception of cross-contamination which was borderline significant (p=0.053). The largest effect was found in equipment cleanliness violations (aOR=0.64).
Full service restaurants were more likely to be cited for all of the targeted critical violations than fast food facilities, with adjusted odds ratios ranging from 1.43 (p=0.015) in sanitizer concentration to 2.12 (p=0.000) in hygiene violations.
Feedback. Results of the assessment have been presented to University of Utah faculty, the SLVHD Board of Health and the Utah Environmental Health Association Fall Conference. The implementation of a web-based system for public access to restaurant inspection results appears to have had some impact on the frequency of critical violations noted by inspectors. It was determined the website is an effective and efficient tool for reducing critical violations cited during restaurant inspections.
Overflow (Objective 2): Please finish the response to the question above by using this text area. Please be mindful of the word limits. Overflow (Objective 3): Please finish the response to the question above by using this text area. Please be mindful of the word limits. What are the specific tasks taken that achieve each goal and objective of the practice? A committee consisting of food inspectors and food bureau supervisors was formed and given the assignment to develop a website which would reduce critical violations by providing restaurant inspection information to the public and restaurant owners or operators. The committee reviewed all existing restaurant inspection websites and identified the features they felt would benefit the website and those which were detrimental to the website purpose. Search and ranking features were identified and evaluated. Once the basic website template was developed, feedback was received from the Utah Restaurant Association. The revised template was then given to the Department’s data contractor for website development and beta testing. Eighteen months after the committee was formed, the website was launched. What was the timeframe for carrying out these tasks? The committee and stakeholders accomplished their tasks within six months. The greatest amount of time was spent in website development and beta testing. The time required to develop and implement an inspection website depends largely on the development contractor. The website could have been implemented sooner if the Department had internal resources to devote to the project rather than contracting with an outside firm for website development. Obviously, the more complex the website, the longer it will take to develop and implement. Because SLVHD included searchable fields and a ranking system, the project took longer to complete. Please provide a succinct outline of some basic steps taken in implementing your practice. The steps taken to implement the website are as follows:
1. Develop committee assigned to determine objectives and purposes of the website.
2. Review other websites, talk with other jurisdictions which have websites, retain the useful functions and discard the less useful elements of those websites.
3. Review draft website design with stakeholders and revise as needed.
4. Deliver draft website design, purposes and elements to the website design contractor for development, beta testing and implementation.
5. Launch website.
6. Evaluate website based on objectives.
7. Contractor maintains website and updates as needed.
What were some lessons learned as a part of your program's implementation process? Implementing an inspection website takes longer than expected. Software development and beta testing were very time consuming. It is also important to plan for a large number of hits when the website is launched. Hits will reduce after the initial launch, but without proper planning, the server may crash. Provide a breakdown of the overall cost of implementation, including start-up and in-kind costs and funding services. The cost to develop, launch and monitor the website is included a data contract cost and is not broken out as a separate item, therefore the exact cost of development and implementation is unknown. However, the cost to design, implement and monitor a website is minimal compared to other programs designed to reduce critical violations. Is there sufficient stakeholder commitment to sustain the practice? Describe how this commitment is ensured. The Utah Restaurant Association continues to monitor the website and make suggestions for its improvement. Most restaurant owners support the website and have learned to use the information to promote their establishment and revise their marketing. The website provides motivation to owners of poor scoring restaurants to improve their inspection scores and possibly increase the number of customers coming to their establishment.
Public input regarding use of the website indicates overall support. The public have provided ideas and comments for improving the website in the future. The Department is exploring the use of QR technology to link restaurant customers to the website after a suggestion from the public. Continued improvement following public comments ensures the website will be a service the public seeks out and utilizes.
The University of Utah is committed to continuing evaluation of the website through additional studies. The Division of Public Health is continually suggesting changes to the website which results in an opportunity to study the effectiveness of those changes in reducing risk. The studies which have been conducted to date have been presented at national conferences and have been accepted for publication in leading journals. This brings notoriety to the University and opportunity for further studies.
Describe plans to sustain the practice over time and leverage resources. The cost of sustaining the website is included in our data maintenance contract. Administration has committed to retaining and improving the website due to its effectiveness in reducing critical violations and public acceptance. Practice Category Choice 1: Food Safety Practice Category Choice 2: Information Systems or Technology Practice Category Choice 3: Environmental Health Check all that apply. Colleague in my health department I am a previous Model Practice Applicant Are you a previous applicant? Yes, and was awarded Promising |
|
|