Summary

Over 364 days the log records 1383 incidents. Findings beyond the team's usual rhythm: 7; the largest are below.

Findings: where and when problems happen

Findings are sorted by size: how many incidents or how much time each gives beyond the team's usual rhythm. First come those where a review can save the most time.

What gives the most incidents and time beyond the usual rhythm

The bar is the share of all incidents or of total incident time beyond the team's usual rhythm. On the right - how much that is over the period.

Finding 1 Size: 30% of total incident time above an even spread, ≈25d 01h 30m

3 of 13 services account for 53% of total incident time

The deviation is statistically significantCause is a hypothesis, to be checked

Why it matters. Most likely, till incident time accumulates from frequent short incidents, so analysing their recurring causes may yield more than analysing the longest outages.

Observation

With an even spread they would account for 23%. The largest is "Store tills": 19d 14h 50m of total incident time.

Evidence
  • 77% of this time comes from priority 3-4 tickets. Ticket time does not always match business downtime: check in monitoring whether the service was really down.
  • Not in the total duration: tickets closed by a timer or a queue clean-up, and open tickets (details where the repair time is described).

3 of 13 services - 53% of incident time

The bar is the service's share of total incident time. On the right - the running total, the bracket marks the group the finding speaks of. The dashed line is the share of each with an even spread.

What it may mean

Most incident time falls on a few services, more than chance would give. A detailed review of these services can give more than equal attention to the whole infrastructure.

Next action

First look into the services with the largest share: 3 of 13. For "Store tills", put its incidents in one sample and look for a common cause instead of the cause of each one.

How to check

Open any three incidents of "Store tills" from the last month. If the causes differ, there is most likely no common problem: the service simply carries more load. If the cause is the same, it is a problem worth tracking separately until it is fixed.

How it was counted, and the log rows
What we looked at

How incidents and their total duration spread across 13 services, and repeats.

Log rows: the longest incidents of “Store tills”

TicketStartServiceObjectLength
INC0260592025-09-10 09:52Store tillsStore #98h 08m
INC0261622025-09-28 15:39Store tillsStore #347h 58m
INC0262362025-10-09 11:08Store tillsStore #407h 33m
INC0260982025-09-18 15:41Store tillsStore #65h 38m
INC0264002025-11-08 08:34Store tillsStore #54h 55m
Does this finding match what you see in practice?
Finding 2 Size: ≈205 incidents above the old pace over 122 days

Around 01.09.2025 the incident rate rose by 52%

The deviation is statistically significantCause is a hypothesis, to be checked

Why it matters. Incidents increased and the frequency has not returned to the previous level, so the cause should be sought in changes within that window.

Observation

It was 3.2 incidents a day, now it is 4.9. The new level has held for 122 days.

Evidence

The incident rate went up: from 3.2 a day at the start of the period to 5.1 at the end. These data cannot tell a step from a slow rise. If it was a step, it came between 13.08.2025 and 24.09.2025, most likely around 01.09.2025: it was 3.2 a day, it became 4.9. If it was a slow rise, there is no single date. Since 01.09.2025 the new level has held for 122 days.

It was 3.2 incidents a day - now 4.9

A dot is a week. The dashed line is the old level, the solid one the new level, the band is where the step may be. On the right - the forecast for 30 days with its likely range.

What it may mean

The journal does not say what happened: it has no cause field. Options: more users or sites, a release or a migration, a new contractor, new logging rules - for example, monitoring started to open tickets by itself. The change log for those weeks and a "registered by" column (person or monitoring) will tell them apart.

Next action

Find what changed in those weeks. If it is business growth, recount the on-call shift for the new level. If it is a release or a contractor, return the question to whoever made the change.

How to check

Look at what you did from 13.08.2025 to 24.09.2025: releases, switch-overs, new contracts, monitoring set-up.

How it was counted, and the log rows
What we looked at

How incidents spread across hours and days of the week, and how the rate changed over time.

Does this finding match what you see in practice?
Finding 3 Size: ≈47 failures above the typical for their systems

Incidents happen more often on 2 objects than on comparable ones

The deviation is statistically significantCause is a hypothesis, to be checked

Why it matters. Specific objects are named, so the analysis can start with them and with their ticket descriptions.

Observation

"ERP-01" - 49 failures against 22 for a typical object of its system; "Store #17" - 29 failures against 9 for a typical object of its system.

Evidence
  • 34% of failures on objects (435 of 1296) came back on the same object of the same system within 7 days after the fix. If every object of each system failed alike, it would be about 33%.
  • Tickets of one outage on one object count as one failure: 87 such tickets.

Failures on "ERP-01" and the other named objects

A row is an object, a mark is a failure. A red dot came within 7 days after the previous fix, a dark tick after a break. On the right: failures on the object against a typical object of the same system, or how many came back.

What it may mean
  • The named objects fail again and again. The same object is not yet the same cause, and the journal does not show the cause.
  • Possible causes:
    • the cause was not removed: it was fixed by a restart or a workaround;
    • the object is worn or overloaded;
    • the object has more equipment or people than its neighbours;
    • one outage is logged again later.
  • What tells them apart: the description and the closing notes of these tickets, a record in the problem log, and how much equipment the object has.
Next action
  • Review the failures of "ERP-01" together, in one review: compare the descriptions and what was done at closing. Then act on what you find:
    • one cause, fixed by a restart - open a problem and look for the cause;
    • different causes, and the object is bigger than its neighbours - compare it with objects of the same size;
    • the same outage logged again - agree to link a repeat ticket to the first one.
How to check

Send the journal again in a quarter: "ERP-01" should have fewer failures and fewer returns within a week.

How it was counted, and the log rows
What we looked at

How incidents and their total duration spread across 13 services, and repeats.

Log rows: failures on “ERP-01”

TicketStartServiceObjectLength
INC0263522025-10-31 16:15ERPERP-012h 21m
INC0265062025-11-27 10:25ERPERP-014h 56m
INC0265092025-11-27 16:21ERPERP-011h 36m
INC0265162025-11-28 12:40ERPERP-013h 34m
INC0267202025-12-31 13:15ERPERP-011h 20m
Does this finding match what you see in practice?
Finding 4 Size: ≈33 tickets of other systems that may follow from one failure

A failure of "Store network links" is followed by tickets of 3 more systems

The deviation is statistically significantCause is a hypothesis, to be checked

Why it matters. If the link is confirmed, a single network link incident affects both payments and tills: such incidents should be restored as a priority.

Observation

After 71 failures of "Store network links", 34 tickets of other systems opened within the first 15 minutes.

Evidence

After a failure of "Store network links", tickets for other systems open within 15 minutes: "Store Wi-Fi" 14 times out of 71, "Store tills" 11 times out of 71, "Payment gateway" 9 times out of 71. By chance, given these systems' own rhythm, it would happen at most 0.7 times over the whole period for each system, fewer than 2 for all of them together.

Which systems fail next

An arrow: within 15 minutes after a failure of the first system a ticket of the second opened. The number is how many times out of all its failures, "chance" is what the usual rhythm gives; the thicker the arrow, the more often. The link in time is confirmed, a shared cause is a hypothesis.

What it may mean

Tickets for "Store Wi-Fi" in the first minutes after a failure of "Store network links" are more likely a consequence than a separate problem. There are three usual reasons: "Store Wi-Fi" works through "Store network links"; both depend on a third thing - power, a shared node, a site; or two teams registered one outage. The object column (store, node, site) tells them apart: if a pair of tickets has the same object, it is one outage.

Next action

Link tickets for "Store Wi-Fi" opened in the first 15 minutes after a failure of "Store network links" to its ticket as child tickets. Then failures stop being counted twice, and people fix one cause. If "Store Wi-Fi" works through "Store network links", "Store network links" needs the backup.

How to check

Open the last three outages of "Store network links" and the tickets for "Store Wi-Fi" in the next 15 minutes: is it the same object, and did they close together with the outage?

How it was counted, and the log rows
What we looked at

Which systems fail together: after a failure of one, a ticket for another opens within 15 minutes more often than chance allows.

Log rows: “Store network links” and right after it “Store Wi-Fi”

TicketStartServiceObjectLength
INC0265752025-12-08 06:20Store network linksStore #442h 41m
INC0265772025-12-08 06:31Store Wi-FiStore #442h 33m
INC0267062025-12-29 21:00Store network linksStore #3013 min
INC0267082025-12-29 21:10Store Wi-FiStore #3017 min
Does this finding match what you see in practice?
Finding 5 Size: ≈30 incidents above the usual rhythm over the period

Thu 02:00-03:00: 34 incidents against about 4 expected

The deviation is statistically significantCause is a hypothesis, to be checked

Why it matters. The hour is known in advance, so the job can be found in the schedule and moved or staggered.

Observation

Failures in this hour came in 31 of 52 weeks; 28 of 34 on Warehouse system.

Evidence
  • By time of the week: weekdays 9 to 18 hold 46% of incidents (27% of the hours of a week), weekday evenings and nights 28%, weekends 27%.
  • Thu 02:00-03:00: 34 incidents, where your usual rhythm would give about 4. Failures in this hour came back in 31 of 52 weeks. Most of them are Warehouse system (28 of 34).

Thu 02:00-03:00: 34 incidents against the usual ~4

The darker the cell, the more incidents. On the right - the total by day, below - by hour. A red frame is an hour that breaks the usual rhythm.

What it may mean

Failures tied to one hour are usually tied to a schedule: a night job (exchange, export, backup), a release window, planned works of a supplier, or a check that writes what it found at its own time. The log alone cannot tell which of these it is.

Next action

Find what runs at this time: the job schedule, the release window, supplier works. If it is a job, move it or add a check before it starts; if it is releases, add a check right after the rollout.

How to check

Open the tickets of Thu 02:00-03:00: if they share one system and similar text, it is one repeating cause. The time is taken as it is in the export. If your system writes time in UTC, shift the hours to your time zone.

How it was counted, and the log rows
What we looked at

How incidents spread across hours and days of the week.

Log rows: incidents in the marked hour

TicketStartServiceObjectLength
INC0263892025-11-06 02:21Warehouse systemWMS-011h 59m
INC0264672025-11-20 02:15Warehouse systemWMS-011h 03m
INC0265052025-11-27 02:23Warehouse systemWMS-012h 04m
INC0266332025-12-18 02:04Warehouse systemWMS-011h 34m
INC0266342025-12-18 02:25Payment gatewayPAY-011h 19m
Does this finding match what you see in practice?
Finding 6 Size: ≈15 failure days above usual

ERP: failures in the last three days of the month - 11 of 12 months

The deviation is statistically significantCause is a hypothesis, to be checked
Observation

Failure days on these dates: 22, the usual rhythm would give about 7.

Evidence

ERP: failures on the last three days of the month - in 11 of 12 months. Failure days on these dates: 22, where the usual rhythm would give about 7. Last time: 30.12.2025. If nothing changes, the next one by the cycle is 29.01.2026-31.01.2026.

ERP: 11 of 12 months with failures on cycle days

Each circle is a day of the cycle: red when the system failed that day, hollow when it passed quietly. Ticks are failures on other days. The hollow red circle after the end of the export is the next day by the cycle, if nothing changes.

What it may mean

Failures that come back at equal intervals are usually tied to a schedule or to something that fills up: a job every few days, month-end closing and reports, disk space or memory that runs out by the same time, a certificate or password that expires. The log alone cannot tell which of these it is.

Next action

Compare these dates with your calendar of month-end closing, reports and payments: what loads the system on these days. Give it resources in advance and put someone on duty who knows the system.

How to check

Open the tickets of ERP on the cycle dates (29.09.2025, 29.10.2025, 28.11.2025, 30.12.2025): if the text is the same, it is one cause. If the system was restarted between failures, the cycle may be the time it takes a resource to run out.

How it was counted, and the log rows
What we looked at

How incidents spread across hours and days of the week.

Log rows: failures of “ERP” on the cycle days

TicketStartServiceObjectLength
INC0261742025-09-29 16:54ERPERP-011h 54m
INC0263382025-10-29 19:44ERPERP-0321 min
INC0265162025-11-28 12:40ERPERP-013h 34m
INC0267112025-12-30 09:47ERPERP-014h 28m
INC0267162025-12-30 15:04ERPERP-014h 14m
Does this finding match what you see in practice?
Finding 7

5 typical tickets repeated 455 times and took 551 h: candidates for automation

Count from the logCause is a hypothesis, to be checked
Evidence

The 5 most frequent texts are 455 tickets, 33% of tickets with a description, 551 h to restore in total: "Till not printing receipts, store #N" - 122 times, 154 h; "Till freezes on scan, store #N" - 104 times, 126 h; "Card payment fails at till, store #N" - 95 times, 106 h; "Fiscal data operator unreachable, store #N" - 86 times, 85 h; "Error on goods receipt" - 48 times, 80 h.

What it may mean

The same text is the same situation that engineers work out again every time. There are three ways to remove this work: an automatic action or a script, an instruction for the first line or for the user, or fixing the common cause.

Next action

For the ticket "Till not printing receipts, store #N", look at how it is closed. The same action every time - automate it or hand it to the first line. Closed with no work - fix the alert threshold. Fixed in different ways - open one problem for the whole series.

How to check

In a quarter: how many times the text "Till not printing receipts, store #N" came and how many hours it took.

How it was counted, and the log rows
What we looked at

Which descriptions in the column "Description" repeat: numbers and addresses are ignored, equal texts are added up.

Does this finding match what you see in practice?

How the service works

The 5% longest incidents give 24% of total incident time

What the data shows
  • Half of the incidents close faster than 54 min, the average is 1h 28m.
  • The worst 5% run longer than 4h 41m, the longest was 19h 30m.
  • Those 69 incidents hold 24% of the total incident duration.

How long incidents last

A bar is how many incidents closed within that time (log scale). The red zone is the 5% longest.

What it may mean
  • The average (1h 28m) is above the median (54 min: half of the incidents close faster). This is usual: a few long cases raise the average.
  • The 5% longest incidents give 24% of total incident time.
  • The median suits reporting better: it shows a typical incident.
Next action

Consider a separate procedure for incidents not restored within 4h 41m: after that time, escalate to an engineer who may bring in the vendor. Agree the threshold with service criticality and your current restore targets instead of taking it from the statistics.

How to check

Open the last three incidents longer than 4h 41m and look at how much time went into diagnosis and how much into the repair itself. If more than half went into diagnosis, the bottleneck is in monitoring, not in the hands of your engineers.

How it was counted, and the log rows
What we looked at
  • Time from registration to restoration for 1369 incidents.
  • This is calendar time: nights and weekends count in full.
  • 14 tickets are not closed: their end is unknown, so they are left out.
  • Priority 1-2 among them: 2, the oldest open since 22.12.2025. If this is an outage going on now, it is larger than any number above.

Log rows: the longest incidents

TicketStartServiceObjectLength
INC0253112025-03-15 22:10ERPERP-0119h 30m
INC0264982025-11-26 06:05Warehouse systemWMS-0115h 00m
INC0257882025-07-08 14:42Online shopWEB-0113h 15m
INC0258072025-07-13 13:43Store Wi-FiStore #3311h 48m
INC0253152025-03-16 19:09BackupBACKUP-0110h 20m

A priority 1-2 incident longer than 6 h: about once a month

What the data shows
  • 2.7% of incidents last longer than 6 h - about one in 37. At your rate that is about 3 times a month.
  • Incidents longer than 6 h among critical and high ones (priority 1-2): 3.4% of them, about once a month.
  • Tickets still open in the export: 14, of them open longer than 6 h: 13. Their end is not known yet, so they are not in the count of long incidents.
  • Priority 1-2 among them: 2, the oldest open since 22.12.2025. Check whether the incident was still going on at the time of the export or the ticket was not closed.
  • If the current rate holds, in the next 7 days expect around 36 incidents (likely range 22-53), over 30 days - around 156 (128-185). This estimate is built from your own incident rate over the observed period, using the last 60 days: the rate changed over the period.

How often long incidents happen

How many times a month an incident lasts longer than the threshold. A threshold longer than the longest incident in the log is a model estimate.

What it may mean

Such a case happens about once a month. It is your own result, not an industry norm: it is counted from your own incidents.

Next action

Write the order of actions for an incident longer than 6 h and run one drill on it: who decides to switch to the reserve, who talks to customers, who talks to the regulator.

How to check

Check whether there is a written plan for a service that is not restored within 3 h: who decides and whom they bring in. If the escalation order is not set in advance, restore time can grow.

How it was counted, and the log rows
What we looked at

The distribution of the longest incidents and the current rate at which incidents appear.

Your key services

What the data shows
  • "Store tills": 407 incidents (30% of all), total duration 19d 14h 50m (24% of the total), half of the repairs faster than 48 min.
  • "Online shop": 163 incidents (12% of all), total duration 5d 09h 49m (6% of the total), half of the repairs faster than 27 min.
  • "Payment gateway": 91 incidents (7% of all), total duration 1d 23h 55m (2% of the total), half of the repairs faster than 12 min.
  • Other services: half of the repairs are faster than 1h 18m.
  • Of the key services, "Store tills" has the largest total incident duration.
What it may mean

The key services are repaired no slower than the rest.

Next action

For "Store tills" - fix the repair time it already holds as a target and watch it on its own.

How to check

In a quarter: the total incident duration and the repair time of "Store tills".

How it was counted, and the log rows
What we looked at

The key services you named: "Store tills", "Payment gateway", "Online shop".

Checked - no deviation

  • There are no more incidents after changes than in the same hours without changes.

What to expect in the 30 days after the export ends

A cell is a day. Red is a cycle day, when a failure is more likely; a frame is a day with a risk hour. Expected incidents over these days: about 156 (128-185).

Action plan

Actions follow the size of the findings. Each stands on a finding above; the result shows in the next report on a quarter of the log.

1First look into the 3 services with the largest share of incident time: review the incidents of "Store tills" together and look for a common cause

Based on: Finding 1. 3 of 13 services account for 53% of total incident time

How to check the result
  • Open any three incidents of "Store tills" from the last month. If the causes differ, there is most likely no common problem: the service simply carries more load. If the cause is the same, it is a problem worth tracking separately until it is fixed.
What helps to be precise

A cause field at incident closing, as a short list

2Find what changed from 13.08.2025 to 24.09.2025: releases, contractors, load, monitoring

Based on: Finding 2. Around 01.09.2025 the incident rate rose by 52%

How to check the result
  • Look at what you did from 13.08.2025 to 24.09.2025: releases, switch-overs, new contracts, monitoring set-up.
What helps to be precise

The change log for these weeks and a "registered by" column (a person or monitoring)

3Review the failures of "ERP-01" together: compare descriptions and what was done at closing

Based on: Finding 3. Incidents happen more often on 2 objects than on comparable ones

How to check the result
  • Send the journal again in a quarter: "ERP-01" should have fewer failures and fewer returns within a week.
What helps to be precise

Ticket descriptions and closing notes, the problem log

4Check a shared dependency on "Store network links" and link the tickets that follow to its outage

Based on: Finding 4. A failure of "Store network links" is followed by tickets of 3 more systems

How to check the result
  • Open the last three outages of "Store network links" and the tickets for "Store Wi-Fi" in the next 15 minutes: is it the same object, and did they close together with the outage?
What helps to be precise

An object column (store, node, site) in the tickets of both systems

5Find what runs Thu 02:00-03:00 on "Warehouse system" and move the job or add a check before it

Based on: Finding 5. Thu 02:00-03:00: 34 incidents against about 4 expected

How to check the result
  • Open the tickets of Thu 02:00-03:00: if they share one system and similar text, it is one repeating cause. The time is taken as it is in the export. If your system writes time in UTC, shift the hours to your time zone.
What helps to be precise

The job schedule, release windows and contractor work windows

6Prepare ERP for 29.01.2026-31.01.2026: an on-call engineer who knows the system, and no changes on these days

Based on: Finding 6. ERP: failures in the last three days of the month - 11 of 12 months

How to check the result
  • Open the tickets of ERP on the cycle dates (29.09.2025, 29.10.2025, 28.11.2025, 30.12.2025): if the text is the same, it is one cause. If the system was restarted between failures, the cycle may be the time it takes a resource to run out.
What helps to be precise

The calendar of period close, reporting and scheduled jobs

7Look at how "Till not printing receipts, store #N" tickets are closed: the same action - automate it or hand it to the first line, no work - fix the alert threshold, different fixes - open one problem

Based on: Finding 7. 5 typical tickets repeated 455 times and took 551 h: candidates for automation

How to check the result
  • In a quarter: how many times the text "Till not printing receipts, store #N" came and how many hours it took.
What helps to be precise

A cause at closing as a mandatory field, as a short list

8Consider a separate procedure for incidents longer than 4h 41m: escalation to an engineer who may bring in the vendor

Based on: The 5% longest incidents give 24% of total incident time

How to check the result
  • Open the last three incidents longer than 4h 41m and look at how much time went into diagnosis and how much into the repair itself. If more than half went into diagnosis, the bottleneck is in monitoring, not in the hands of your engineers.

To make the next report more precise

  • Count failure numbers only on records with the incident type in the column “Issue Type”, and make sure the type is always filled.
  • Make the cause field mandatory at closing and fill it from a short list of values instead of free text: a short list of causes makes filling consistent and later analysis easier. In a quarter you can compute the share of incidents with a known root cause - today there is no data for it.

What we did not get

These questions help to see what you can do as the next steps.

Do you measure resource load, losses, delays or response time? We did not get a metrics export.

Do you keep recurring causes separately, as problems?

Do you keep a change log? We need the date and time of each change.

Operations maturity and the next step

This rates practices, it does not measure failures: the step says what the team already does and which step to the next level is closest.

Step 2 of 5 - "Recording"

Outages are recorded. That history already shows how long recovery takes, where incident time piles up and what outages to expect next.

The step is counted from the log only. Functions the log does not show are not counted yet. So for now the result is at the lower edge of the step.
Confirmed by data: 1 of 1 key functions up to step 2.

What to do for step 3 - "Managing"

This is what the log did not show and what we asked you to add. If you already have some of it and it just did not get into the export, your step is higher, and you can skip these tips.

Set a recovery time for each priority, at least for the main services, and agree it with the consumer.
What it gives: you and the consumer have one answer to "are we on time", and for every outage you know whether it was closed in time.
Put someone on call for every shift and write down who takes the outage over if nobody picked it up or it was not restored within part of the promised time.
What it gives: you do not lose time, and you control how the work on an outage goes.

Key functions found in the data you sent

StepFunctionITIL 4 practiceStatus
2 Every outage is recorded Incident management confirmed by data
3 Main services have owners Service level management confirmed by data
3 Priority is set by impact and urgency Incident management confirmed by data
3 Recovery times are promised Service level management not seen in the data
3 An on-call rota and a hand-over order Service desk not seen in the data
4 Repeats are investigated down to the cause Problem management not seen in the data
4 Changes are recorded with a type and a result Change enablement not seen in the data
4 Outages are known before the consumer notices Monitoring and event management not seen in the data
5 Improvements have an owner and a measured effect Continual improvement not seen in the data
5 Measures are reviewed with the consumer Measurement and reporting not seen in the data
5 Service dependencies on equipment are known Service configuration management not seen in the data
What this is based on

The assessment scheme follows formal process assessment (ISO/IEC 33000, the CMMI appraisal method): what people say and what the work records show. An answer makes a function "declared", the export makes it "confirmed", a mismatch makes it "disputed".

A function is confirmed by data when the log covers at least 90 days and 30 records, the start, end and object of an outage are filled in 95% of records, the function's own field (service, priority) in 80%, and no single value stands on more than 95% of records.

Practices and their names come from ITIL 4. The ladder rests on ITIL 4, CMMI and ISO/IEC 33000; it is not a certification and not an assessment by an accredited assessor.

What the steps mean

Appendix A. Report data

Sources

This report is built only on these sources:

SourceWhat it held
Incident journalAn export from your ticket system for the period 364 days, 1728 records. Fields: time registered, time closed, service or node, priority, description or cause.
List of critical servicesAs named by you: Store tills, Payment gateway, Online shop.
Metric exportsNot uploaded. We need CPU, memory and link load, packet transfer delay, packet loss ratio, delay variation, backup success and certificate expiry - a measurement time and a value column for each one. (see the check map below)
Change logThe incident journal has 25 records of the type "change" or "planned maintenance". There are no more incidents after changes than in the same hours without changes.
First-line support dataNot uploaded. We need the resolving group or support line for each ticket, an escalation flag, and a reopen flag. (see the check map below)

What the log holds: failures and the rest

How 1728 records became 1369 incidents

Every number of the report is counted on one of these steps: rate and links on failures with the open ones, time and downtime on closed ones, repeats on failures on objects.

What we looked at. 1728 journal records over 364 days.

What we found.
  • Out of 1728 journal records, 359 (21%) did not go into the incident analysis:
    • 275 - service requests by the column “Issue Type”: they are not failures;
    • 45 - service requests too: their type is empty, we recognised them by the words of the description (an estimate);
    • 25 - changes, problems or planned works;
    • 14 - tickets not closed: the end is unknown, so the duration cannot be counted.
  • Below we work with the remaining 1369 incidents - 3.8 per day.
  • Downtime - the time when at least one incident was running: 69d 12h 24m. Parallel incidents count once here; the total incident duration, where each counts on its own, is 83d 06h 19m.

What it means. Counted all together, there are 25% more failures than really happened, and the repair time mixes with the time to fulfil requests. A management report on all tickets shows more outages than there were.

What to do. Count failure numbers only on records with the incident type in the column “Issue Type”, and make sure the type is always filled.

Check it yourself.* Filter the export by the column “Issue Type”: you should get 300 records typed as a request, change or problem.

Examples of log rows we took out

TicketStartType or description
INC0250002025-01-01 08:54:09Service request
INC0250022025-01-01 12:51:31Service request
INC0250122025-01-03 11:41:38Service request
INC0250622025-01-17 10:02:51Unlock my account after holiday
INC0250762025-01-20 16:48:40Please install the e-signature software

Data quality

What we looked at. The completeness and consistency of the journal itself.

What we found. Data quality grade - B (out of A, B, C, D). Field completeness:

What it means.
  • The fields time registered, time closed, service or node, priority, description or cause are filled well - so everything we said based on them is reliable.
  • Records with no closing time were excluded from the duration calculation, not given an average value.

What to do. Make the cause field mandatory at closing and fill it from a short list of values instead of free text: a short list of causes makes filling consistent and later analysis easier. In a quarter you can compute the share of incidents with a known root cause - today there is no data for it.

Check it yourself.* Open your ticket system and look at which fields are mandatory at closing. If the cause is not mandatory, that is why it is not in this report.

FieldFilled
Time registered100%
Time closed99%
Service or node100%
Priority100%
Description or cause100%

Notes on the numbers

  • 80 tickets out of 1383 (incidents with the open ones) opened in the first 15 minutes of an outage already under way: they continue it, or register the same outage again. The number of failures and the time between them in this report are counted in tickets, so they are overstated by about 6%; in the list of the largest incidents such an outage counts as one.

Appendix B. Full check map

The map holds every kind of check we offer today. Some checks may need extra data from you (see "Details" in the row).

What the review covers

CoverageWhat it gives
Computed from the log7 findings and 6 team figures - everything above and below in this report.
Can be refined with your fields and targets11 more: availability of critical services, error budget spent, actual recovery time against the target, time to detect, time to respond and 6 more.
Other sources needed3 more: a metric before failures, share of emergency changes, capacity and health.
Does the infrastructure keep the promised level - 1/4
Downtime (time when at least one incident ran)Details

What this is. Total time your services were unavailable over the log period.

How we count it. We add up the duration of all incidents, but merge overlapping intervals: two incidents running at the same time from 10:00 to 11:00 count as one hour of downtime, not two. Without merging, the number is inflated more the larger the infrastructure is.

Where the definition comes from. Our calculation. The concept of downtime comes from availability in the ITIL 4 glossary: the ability of a service to perform its agreed function when required.

What we need from you. Already calculated - see the number in the row.

If the row is amber: we need a field with the incident closing time, otherwise the duration cannot be computed.

69d 12h 24m
Availability of critical servicesDetails

What this is. The share of observation time the service was up. This is the number people call "three nines".

How we count it. Uptime / (Uptime + Downtime), calculated separately for each service you named. An incident that hit three services counts in full for each of the three: from that one service's point of view, it was down the whole time.

Where the definition comes from. Formula - Google SRE Book, ch. 3 "Embracing Risk", verbatim. The concept of availability - ITIL 4 glossary.

What we need from you. Mark the critical services in the list and state the target level (e.g. 99.9%). We calculate right after you answer - no need to re-upload the report.

Critical services The names come from the "service" field of your journal, as they are. If that field holds devices or device models, availability is counted for the device, not for the service it carries.

choose your critical services and your target level
Error budget spentDetails

What this is. A target like 99.9% allows a certain amount of downtime by definition. That allowed time is the budget. This indicator shows how much of the budget you have already spent.

How we count it. Allowed downtime = period length x (1 - target). Spent = actual downtime / allowed downtime x 100%. We use the actual period - exactly as many days as are in your log, with no rounding to a month. A value above 100% means the budget is used up and the target was missed for the period.

Where the definition comes from. Google SRE Book, ch. 3: the budget is the gap between the target and the actual uptime - the remaining "unreliability" for the period.

What we need from you. One number - the target availability. Without a target the indicator does not exist: the budget is counted from a promise, not from the data.

set your target - without it the measure does not exist
Actual recovery time against the targetDetails

What this is. RTO (Recovery Time Objective) is the deadline for a service to come back after an outage. This indicator shows how often you met that deadline.

How we count it. The share of incidents no longer than your target, plus the worst case for the period - the longest incident. The worst case matters more than the average: a continuity plan is tested on a bad day, not on a normal one.

Where the definition comes from. ISO 22300:2021 (vocabulary for ISO 22301): RTO is "the period of time following an incident within which a product, service or activity is resumed, or resources are recovered". The standard is paid, so we have not verified the clause number and do not cite one.

What we need from you. Target recovery time in hours. If it differs by service, give the target for the most critical one.

set your target - without it the measure does not exist
How fast you repair - 1/6
Time to restore a serviceDetails

What this is. How long it takes from an incident being logged to being closed.

How we count it. We give three numbers at once: median, mean, and P95. The median is a normal day: half the incidents are fixed faster. The mean is pulled up by rare, long cases. P95 is the "tail": only one incident in twenty takes longer. A single average without the tail hides exactly the cases that make the business call.

Where the definition comes from. ITIL 4 calls this MTRS, mean time to restore service: "a measure of how quickly a service is restored after a failure". The reliability vocabulary IEC 60050-192 calls the same number MTTR (term 192-07-23).

What we need from you. Already calculated.

If the row is amber: we need a field with the incident closing time.

54 min / 1h 28m / 4h 41m - median / mean / P95
Time to detectDetails

What this is. How long an incident ran before you knew about it. This is a blind spot: the service is already down, but there is no ticket yet.

How we count it. Actual incident start minus the time it was logged in the system. This cannot be derived from the logging time alone under any assumption: the log knows when the problem was reported, not when it began.

Where the definition comes from. No standard exists. MTTD is an established incident-management practice, and we say so plainly instead of dressing practice up as a norm.

What we need from you. A target will not fix this - we need an export with a separate "actual start" field (systems name it differently: start time, occurred at, time of occurrence). Upload such an export via the same link, and the row will be calculated.

needs the field "real start time, separate from the time of registration" filled in your database
Time to respondDetails

What this is. How long a ticket waited for someone to pick it up. This shows how the queue and the on-call shift are performing, not how hard the fault was.

How we count it. Time work started minus the time the ticket was logged.

Where the definition comes from. No standard exists. MTTA is a service-desk practice.

What we need from you. We need an export with a "work started" field (picked up, assigned, acknowledged - whatever your system calls it).

needs the field "time accepted by an engineer" filled in your database
Share closed on timeDetails

What this is. What share of incidents met the deadlines agreed with the business.

How we count it. Calculated separately per priority: how many incidents of that priority closed within your target. Priorities cannot be combined into one number - their targets differ, and a combined figure would hide overruns on the critical ones. We only calculate this when priority is filled in for at least 80% of records; below that threshold the number reflects log data quality, not service performance.

Where the definition comes from. ITIL 4: service level - metrics of expected or achieved quality; SLA - a documented agreement on required services and the expected level of service.

What we need from you. Target times per priority, in minutes (e.g. critical - 60, high - 240, normal - 1440).

If priority is not tracked in the log, or is filled in for less than 80% of records: we first need an export with this field - a target has nothing to apply to without priority.

set your target - without it the measure does not exist
Share solved the first timeDetails

What this is. What share of incidents did not reopen after being closed. A direct measure of repair quality: was the ticket closed, or was the problem solved.

How we count it. The share of incidents with no reopening marker.

Where the definition comes from. No standard exists - this is service-desk practice.

What we need from you. We need an export with a "reopened" field, or a status-change history - from the history we can reconstruct the fact of reopening ourselves.

needs the field "reopened" filled in your database
Share solved on the first lineDetails

What this is. What share of incidents were closed without escalating to specialists. Shows how much work moves upward and how loaded the expensive hands are.

How we count it. The share of incidents closed by the same group that took them in.

Where the definition comes from. No standard exists. ITIL 4 defines only escalation and service desk; the metric itself is not in the glossary.

What we need from you. We need an export with a "resolver group" or "support line" field.

needs the field "resolver group" filled in your database
How often and what exactly breaks - 4/9
How often incidents happen and how often a service failsDetails

What this is. How often something breaks in the whole company and how often each of the most frequent services fails.

How we count it. The period divided by the number of incidents: for the company - the overall rate, for a service - the time between failures. We count service failures, not hardware reliability.

Where the definition comes from. ITIL 4, MTBF: "a measure of how frequently a service or other configuration item fails". IEC 60050-192, term 192-05-13, mean operating time between failures.

What we need from you. Already calculated.

1369 incidents, on average one every 6h 23m.
Most often: Store tills - once every 21h 29m; Online shop - once every 2d 05h 38m; Mobile app - once every 2d 23h 04m
Concentration of incident duration by serviceDetails

What this is. Shows whether your incident time is concentrated in a few points or spread evenly. If concentrated, work on a short list gives a disproportionately large effect.

How we count it. We rank services by their share of the total incident duration and check how much of it the top ones account for. We also calculate an inequality score between 0 and 1 (named in Appendix A for readers who want the formula): 0 means "all services contribute equally", closer to 1 means "almost everything falls on a few".

Where the definition comes from. Our calculation. No borrowed norms exist: we compare your infrastructure with itself, not with someone else's average.

What we need from you. Already calculated.

If the row is amber: we need a "service or node" field - without it incident time cannot be attributed. If grey-blue: this one comes with the extended review.

3 of 13 services - 53% of total incident time
Returns to the same objectDetails

What this is. What share of failures came back on the same object - server, CI, store - within a week after the fix, and whether that is more than chance gives.

How we count it. The key is the object together with its system: "the till in store 12", not "tills". Tickets of one outage on an object (opened while the previous one is open, or within an hour after it closed) count as one failure. A return is a failure within 7 days after the end of the previous one. Next to it: the share chance would give if all objects of a system failed alike. An object is named when it fails more often than the other objects of its system, or its failures come in series more often than the others' rhythm gives; both tests are corrected for the number of objects. The same object is not yet the same cause: in ITIL 4 a repeat points to a problem, "a cause, or potential cause, of one or more incidents".

Where the definition comes from. Our calculation; interpretation - ITIL 4, problem.

What we need from you. Already calculated.

If the row is amber: we need an object column - server, CI or store - filled in at least one ticket of five. If grey-blue: available in the extended review.

34% of failures came back on the same object within 7 days after the fix; if objects failed alike, about 33%
A metric before failuresDetails

What this is. Which monitoring metric - CPU load, memory, traffic, disk - moves away from usual before failures, and about how long before them.

How we count it. We put the journal and the metrics export on one time axis. "Usual" for a metric is its level at the same hour of the day, workdays and weekends apart. We look at windows 15 min - 1 h, 1-6 h and 6-24 h before the ticket and compare how often the metric was above or below usual before failures and at the same hours on other days. A finding needs this to happen clearly more often before failures, on different days, corrected for the number of series. A ticket opened by an alert on the same metric is not a precursor.

Where the definition comes from. Our calculation; tested on artificial months and years of monitoring with a known answer.

What we need from you. Already calculated.

If the row is grey: send a monitoring metrics export for the same period together with the journal. If amber: the common period or its failures are too few - it needs 14 days and 8 failures or more.

needs another data source: a monitoring metrics export for the same period (Zabbix, Grafana, Prometheus)
What stands behind the failures needs the field "failure cause (a short list)" filled in your database
Breakdown by priorityDetails

What this is. How incidents are distributed across importance classes.

How we count it. We calculate the share of each priority, but only if the field is filled in for at least 80% of records. Below that, the breakdown reflects log-filling discipline, not the real picture of breakages, and we do not print it. Look at two things. First, the share of the top classes: if critical is more than a quarter of the flow, priority is set by how loud the reporter is, not by business impact, and the urgent lane stops being urgent. Second, compare it with "how fast you repair": critical incidents repaired no faster than normal ones mean the priority exists in the log but not in the shift's work.

Where the definition comes from. ITIL 4, incident: an unplanned interruption to a service or a reduction in its quality. The standard does not define priority scales - we use yours.

What we need from you. Already calculated.

If the row is amber: we need an export with a "priority" field, or for it to be filled in.

P1 - CRITICAL - 53 (4%);
P2 - HIGH - 270 (20%);
P3 - MEDIUM - 667 (48%);
P4 - LOW - 393 (28%)
Share of incidents with a known root causeDetails

What this is. What share of incidents were traced to a root cause, rather than just closed after the service was restored.

How we count it. The share of records with a root cause filled in, or linked to a problem record.

Where the definition comes from. ITIL 4: problem - the cause, or potential cause, of incidents; known error - a problem that has been analysed but not resolved (a workaround is not required for this).

What we need from you. This field is not tracked in your log. That fact alone is already a finding: repeat incidents are closed without root-cause analysis, so they will return. To see a number, we need an export with a "root cause" field, or a link between incidents and problems.

the field is not kept - probably repeat incidents are closed without anyone finding the cause
Share of emergency changesDetails

What this is. What share of changes are made in "needed it yesterday" mode, without the usual approval. A high share means the planned process cannot keep up with reality.

How we count it. The share of emergency changes out of all changes in the period.

Where the definition comes from. ITIL 4: emergency change - a change that "must be implemented as soon as possible"; standard change - a pre-authorised, low-risk change.

What we need from you. A field in the incident log will not fix this: the denominator here is changes, not incidents. We need an export from the change log for the same period. It will also refine the "incidents with a change trace" row - currently that row is a lower-bound estimate from description text.

needs another data source: a change log
Share of incidents caused by suppliersDetails

What this is. What share of breakages came from outside your perimeter - a provider, vendor, or contractor. This is a number for a conversation with the supplier, not with your team.

How we count it. The share of incidents with an external party named as at fault.

Where the definition comes from. ITIL 4, supplier management practice.

What we need from you. We need an export with an "at-fault party" or "supplier" field.

needs the field "party at fault" filled in your database
Capacity and health
This is where the state of your resources belongs: load, network delay and loss, backup success, certificate expiry, out-of-support components. An incident journal does not hold them - we need a separate metrics export for the same period: the time of each measurement and one value column per measure.

Legend: measuredneeds your field/targetneeds another data sourceavailable in the extended review

In total: 6 measures computed, 11 wait for your decision or a filled field, 8 need other data.

Every line marked "needs your field/target" is a chance to make the report better. It is the name of a field somewhere in your database.

Appendix C. Methods applied

We left method names out of the main text on purpose, to keep it light.

What in the reportHow it was computed
DowntimeTime when at least one incident ran: overlapping intervals are merged before summing, otherwise parallel incidents are counted twice
Time to restore a serviceMedian, mean and 95th percentile of durations. ITIL 4 calls this measure MTRS (mean time to restore service); in the reliability vocabulary IEC 60050-192 the same number is called MTTR
Time between failuresFor the whole company - the period divided by the number of incidents: how often something breaks. For a service - the period divided by its incidents: this is the time between failures (ITIL 4: MTBF). We count service failures, not hardware reliability
Concentration of incident durationSumming incident duration by service. Separately, the Gini coefficient: how differently the incidents themselves last. Yours is 0.53: 0 means every incident lasts the same, above 0.6 means a few incidents give most of the duration
Returns to the same objectThe same object of the same system within 7 days after the fix; tickets of one outage are one failure; compared with chance where the objects of a system fail alike
Share of service requestsBy the ticket type column where it exists and is filled; otherwise by normal service phrases in the description (grant access, reset a password, create an account) with no words about a breakdown next to them. The records found are taken out of every calculation; what is found by words is an estimate from below
Links between systemsA ticket of another system within 15 minutes after a failure, against that system's own rhythm (level over the weeks around, weekday, hour); corrected for the number of pairs tested (247); a link must hold without its two busiest days; days of common storms left out
What stands behind the failuresA count by your cause column: cases and the sum of durations for each cause; empty, "other" and "unknown" are not counted. Descriptions: numbers and addresses are ignored, equal texts are added up; texts with an e-mail, a phone or a signature are not shown
Rate change on 01.09.2025Weekly means before and after every possible date, corrected for the week-to-week spread; signal strength 5.4 against a threshold of 3.3 (ordinary years cross it in about 3-5% of cases)
Probability of a long incidentGeneralized Pareto distribution over threshold exceedances (Peaks-Over-Threshold). Shape parameter ξ = +0.12, 95% confidence interval [-0.07, +0.27]; 137 exceedances of the 3.4 h threshold
Rate forecastMean rate of the current regime, no trend carried forward; the prediction interval follows the spread of your weeks and months (Poisson or, when failures come in waves, negative binomial)
Data quality gradeCompleteness of mandatory fields, consistency of times, share of excluded records

Lorenz Curve - Concentration of Incident Duration

The curve pulls clearly away from the straight line - confirmation that incidents last very differently: 41.5% of them hold 80% of the total duration (Gini=0.534). A straight line would mean every incident lasts the same.

Brief Glossary of Methods Used

p-valueProbability of observing results at least as extreme as measured, assuming no real effect. p < 0.05 is statistically significant (< 5 % chance it is random).
95% CI95 % Confidence Interval - the range that contains the true value in 95 out of 100 repeated experiments. Wider CI = more uncertainty.
Gini coefficientMeasures how unevenly the total incident duration is spread over incidents. 0 = all incidents equal; 1 = one incident gave all of it. Gini > 0.6 = strong concentration.

Appendix D. Terms

The wording follows the official ITIL 4 glossary.

TermMeaning
EventAny change of state that has significance for the management of a service
IncidentAn unplanned interruption to a service or reduction in the quality of a service
ProblemA cause, or potential cause, of one or more incidents
Known errorA problem that has been analysed but has not been resolved
WorkaroundA solution that reduces or removes the impact of an incident while a full resolution is not yet available
ChangeThe addition, modification, or removal of anything that could affect services
Service requestA request from a user for an action agreed as a normal part of service delivery. Not an incident
Time to restore a serviceHow quickly a service is restored after a failure
Incident durationTime from opening to closing of one incident. Its typical value (median) is the repair time
Total incident durationThe durations of all incidents added up; parallel incidents each count. Shares of services and causes are taken from it
DowntimeTime when at least one incident was running; parallel incidents count once

* These tasks can be done more precisely and to a higher standard with our help.

What else we can do: walk through this report with you, run a health-check of your service (a one-off audit: where the downtime is, what changed, what to expect next) or research your data deeper around your own question. This is paid work by an OpsLab engineer, by agreement; write to hello@opslab.consulting.

The source log this report is built from

First 12 rows of 1728. The whole file is below.

TicketStart timeEnd timeServicePriorityStatusAssignment groupDescriptionConfiguration itemIssue Type
INC0250002025-01-01 08:54:092025-01-01 11:52:10EmailP3 - MediumClosedInfrastructureNew laptop for a new hireMAIL-01Service request
INC0250012025-01-01 11:18:422025-01-01 14:10:00ERPP3 - MediumClosedERP teamExchange with online shop failsERP-03Incident
INC0250022025-01-01 12:51:312025-01-01 19:16:48WorkplacesP4 - LowClosedService deskNeed access to CRM for a sales managerStore #0Service request
INC0250032025-01-01 14:31:402025-01-01 16:04:06Store tillsP3 - MediumClosedService deskTill not printing receipts, store #42Store #42Incident
INC0250042025-01-02 00:19:412025-01-02 00:55:00Warehouse systemP3 - MediumClosedWarehouse ITHandheld scanners out of syncWMS-02Incident
INC0250052025-01-02 02:20:362025-01-02 04:11:28Warehouse systemP3 - MediumClosedWarehouse ITError on goods receiptWMS-01Incident
INC0250062025-01-02 09:40:382025-01-02 09:47:16Mobile appP2 - HighClosedWeb teamApp cannot open the cartAPP-02Incident
INC0250072025-01-02 14:36:352025-01-02 15:09:50Online shopP3 - MediumClosedWeb teamCatalogue loads slowlyWEB-02Incident
INC0250082025-01-02 15:03:462025-01-02 18:01:30Store network linksP4 - LowClosedNetwork teamPacket loss on store #11 linkStore #11Incident
INC0250092025-01-03 08:00:242025-01-03 09:09:03Store tillsP4 - LowClosedService deskFiscal data operator unreachable, store #9Store #9Incident
INC0250102025-01-03 08:06:202025-01-03 10:40:09Online shopP4 - LowClosedWeb teamCatalogue loads slowlyWEB-03Incident
INC0250112025-01-03 10:22:262025-01-03 11:00:14Online shopP2 - HighClosedWeb teamCatalogue search is downWEB-01Incident

Download the whole log (CSV)

Four columns without names are enough: ticket number, start date, end date and service. The other columns make the analysis more detailed, but are not required.

↑