OKR stands for Objectives & Key Results.
KPI stands for Key Performance Indicator.
It never ceases to amaze me how many public authorities and organizations want to implement digital accessibility without concrete objectives and measurable results. That’s why I want to cover the topic in more depth over the coming days. I explain in general terms what OKRs are, what they are good for, what makes good OKRs and which typical mistakes people make. I also present practical examples of good and bad OKRs in digital accessibility.
What are OKRs (Objectives & Key Results) for?
OKRs help everyone involved know what matters most during a given period. Team leadership sets a small number of clearly defined and prioritized objectives. These are meant to be achieved within, for example, a quarter or half a year. There is a clear baseline and a measurable target. These measurements are repeated at regular intervals. This makes it possible to see the actual progress, to make quick corrections when problems become apparent, and to learn from them. OKRs also ensure that objectives and progress are transparent and are discussed regularly by everyone involved. This helps teams align with the shared objectives, learn from each cycle and improve their approach together.
What is the difference between OKRs and KPIs (Key Performance Indicators)?
OKRs are a strategic management tool. They describe progress within a set period and contain both qualitative objectives and measurable results. KPIs, on the other hand, describe the current state or performance of a system. They are usually measured continuously, for example the error rate in a development system.
So OKRs define a direction and the desired target state, while KPIs indicate the state at any given moment.
In some cases, KPIs can be used as metrics for the key results of OKRs in order to describe the desired changes. However, you have to make sure that the focus is on state and benefit (outcome), not on general activities.
How do you recognize high-quality OKRs?
- The objectives are meaningful and worded so that everyone involved truly understands them.
- The objectives describe concretely which state should change, not an activity.
- The objectives can realistically be achieved within the given period (e.g. a quarter or half a year).
- The key results can be measured and objectively verified. They have a concrete starting value and target value.
- The key results are geared toward real impact (outcome), not activities (output). Outcome means the actual benefit or impact for users or the organization. Output only means the amount of work produced (e.g. number of training sessions, number of tickets, number of meetings, documents created).
- The team can achieve the key results through its own actions, or at least influence them significantly.
- The number of objectives and key results is limited (for example 2 to 3 objectives per team/cycle, each with 3 or 4 key results).
- The objectives and key results are known to everyone involved, easy to find, accessible and available to read. Responsibilities are clearly defined and match the skills of the people/teams they have been assigned to.
- The objectives and key results are reviewed together regularly and progress is discussed.
- The objectives and key results clearly contribute to a positive benefit for disabled users and the organization.
What are typical mistakes with OKRs?
- Tasks were defined instead of objectives.
- The key results don’t define any concrete benefit/added value but are focused on activities.
- They are not clearly measurable, or essential information is missing (e.g. starting value and target value).
- They are unrealistic, i.e. not achievable with the resources actually available and the defined priorities.
- They are unambitious and therefore don’t lead to positive development/real benefit.
- Far too many objectives/results were set, making it impossible to work toward them with focus.
- They don’t fit the organization’s defined overarching objectives or were not embedded in them.
- As a result, they are not taken seriously, don’t get the necessary priority/resources and send contradictory signals to everyone involved.
- The people involved cannot influence them positively themselves, so they don’t feel responsible for them/the OKRs have no relevance for them.
- Once set, they are not measured regularly and systematically and not discussed with the people involved. There should be fixed routines (e.g. regular reviews and retrospectives) in which the OKRs are used as an active working tool.
- There is no system for how to deal with missed results and learn from them.
- They are tied to bonuses or penalties, creating a strong incentive to produce sham OKRs or falsify results. OKRs should primarily be understood as a learning and management tool, not as a pure controlling tool.
- They were not developed and communicated together with everyone involved, but simply imposed from above. Often the people involved know them only superficially or not at all.
Examples of bad or immature OKRs for digital accessibility
Example objective: Improve accessibility
Key results: Carry out an accessibility audit, fix 50% of WCAG errors, provide accessibility training for 100% of developers.
Bad OKR because: Activities are confused with achieved benefit here. Audits carried out, blanket bug fixes or training sessions are activities, not a concrete result. The objective is too vague and general. This does not ensure that the result really is better accessibility for disabled users.
Example objective: Web Content Accessibility Guidelines (WCAG) implemented
Key results: 100% score in the automated accessibility tool, all WCAG 2.1 AA errors in the automated tool fixed.
Bad OKR because: Automated tools can only reliably detect a small percentage of the WCAG criteria. A result of 100% therefore does not mean that the web application is really accessible. Even if the result of an automated tool is perfect, there could still be significant barriers for disabled users.
Example objective: Avoid lawsuits over accessibility violations
Key results: Respond to all official complaints within 7 days and resolve them within 30 days.
Bad OKR because: Here, accessibility is obviously only dealt with once there are already problems and they have been reported through official channels. This is ethically very problematic and the most expensive way to deal with the topic. If accessibility is instead built into all planning and development from the start as a fundamental quality characteristic, much better results are achieved.
Example of a good OKR for digital accessibility
Objective: “People with motor disabilities and blind people can use the entire online application process independently and without help from others.”
Examples of key results:
- The central end-to-end user flow (home page, registration, application form, confirmation) is successfully completed
by most participants with motor disabilities in the usability test held every 3 months.
Starting value: currently 65%.
Target value: at least 90%. - The number of critical barriers (blockers) that blind users with a screen reader report in the corresponding usability test on
the same process drops from 5 to 0.
Starting value (last test result): currently 5 blockers.
Target value: 0 blockers. - The overall share of users with disabilities who state in the survey during the usability test held every 3 months that
they abandoned the process completely because of a barrier drops significantly.
Starting value: currently 35%.
Target value: below 15%.
This is a good OKR because it focuses on directly noticeable benefits for users with disabilities, aiming for fewer abandonments and less frustration. These are checked realistically in regular usability tests with real users. Concrete starting and target values have been specified.
One last important point:
Objectives and key results must match the maturity level of the authority or company. That’s why it is important to determine the current status at the start:
- Maturity level of the authority/organization with regard to digital accessibility
- Current status of the relevant digital applications (websites, intranet, software, apps, hardware, documents)
- and possibly also the individual maturity level of selected departments and teams
Only once this information is available can objectives and key results be aligned with it. Otherwise you stumble straight into one or more of the problems described at the beginning, where the objectives are unrealistic because, for example, the team is only just starting out.