Post processing

PostProcessor

A type to provide data processing features supplying reports, plots, or other data analyses.

Internal Fields

• simulation::Simulation: Simulation object
• infectionsDF::DataFrame: Infections (joined with popuation to get information on infeter and infectee)
• populationDF::DataFrame: Population dataframe with one row per individual
• vaccinationsDF::DataFrame: Output of the vaccination logger
• deathsDF::DataFrame: Output of the death logger
• testsDF::DataFrame: Output of the test logger
• pooltestsDF::DataFrame: Output of the pool test logger
• quarantinesDF::DataFrame: Output of th quarantine logger
• cache::Dict{String, Any}: Internal cache to store and retrieve intermediate results

PostProcessor(simulation::Simulation)

Create a PostProcessor object for an associated Simulation. Post Processing requires a simulation to be done.

PostProcessor(simulations::Vector{Simulation})

Create a vector of PostProcessor objects for a vector of associated Simulation objects. Post Processing requires all simulations to be done.

PostProcessor(batch::Batch)

Create a vector of PostProcessor objects for all Simulations in a Batch. Post Processing requires all simulations to be done.

age_incidence(postProcessor::PostProcessor, timespan::Int64, basesize::Int64)

Returns a DataFrame containing the infection incidence stratified by (10-year) age groups. The timespan parameter defines a time window in ticks and the basesize parameter defines the reference population size to calculate incidences. Assuming 1 tick to be 1 day, a timespan = 7 and basesize = 100_000 configuration results in the commonly known seven-day incidence per 100,000.

The structure assumes no individuals exceeding the age of 100.

Parameters

• postProcessor::PostProcessor: Post processor instance
• timespan::Int64: Reference time window to calculate incidence
• basesize::Int64: Reference population size to calculate incidence

Returns

• DataFrame with the following columns:

age_incidence(rd::ResultData)

Returns a DataFrame with incidence over time stratified by age groups. Look up the PostProcessor docs to find the column definitions. Returns an empty dictionary if the data is not available in the input ResultData object.

attack_rate(postProcessor::PostProcessor)

Divides the number of individuals who have been infected one (or multiple) time(s) by the total number of individuals.

compartment_fill(postProcessor::PostProcessor)

Returns a DataFrame containing the total count of infected individuals in the respective disease states exposed, infectious, deceased and recovered.

Returns

• DataFrame with the following columns:

compartment_periods(postProcessor::PostProcessor)

Calculates the durations of the disease compartments of all infections and returns a DataFrame containing all additional infectee-related information.

Returns

• DataFrame with the following columns:

cumulative_cases(postProcessor::PostProcessor)

Returns a DataFrame containing the cumulative infections count of infected individuals in the respective disease states exposed, infectious, and removed.

Returns

• DataFrame with the following columns:

cumulative_deaths(postProcessor::PostProcessor)

Returns a DataFrame containing the total count of individuals that died.

Returns

• DataFrame with the following columns:

cumulative_disease_progressions(postProcessor::PostProcessor)

Calculates the accumulated number of individuals in a certain disease state (latent, presymptomatic, symptomatic and asymptomatic) after the individual has been infected. Rows indicate the number of elapsed ticks since infections.

Example: Row 8 showing [20, 47, 290, 50] would mean that eight ticks after exposure, 20 individuals were latent, 47 were presymptomatic (no symptoms yet, but will be developing), 290 had symptoms and 50 are not experiencing symptoms and won't ever do.

Returns

• DataFrame with the following columns:

cumulative_quarantines(postProcessor::PostProcessor)

Returns a DataFrame containing cumulative information about days spent in isolation.

Columns

deathsDF(postProcessor::PostProcessor)

Returns the internal flat deaths DataFrame.

Columns

detected_infections(postProcessor::PostProcessor)

A subset of the infections dataframe with only the detected cases. An infection is considered to be detected if the infection has a non-missing first_detected_tick. For the column definitions, look up the infectionsDF(postProcessor) documentation.

Returns

• DataFrame: Please look up the column definitions in the infectionsDF(postProcessor) documentation.

detected_tick_cases(postProcessor::PostProcessor)

Returns the number of detected cases per tick and the number of true new exposures. This analysis is based on the tick_test column of the infections-dataframe which indicates when an individual with an active infection was first tested positive. Thus, there might be a delay between exposure_cnt and detected_cnt.

Returns

• DataFrame with the following columns:

detection_rate(postProcessor::PostProcessor)

Returns the fraction of detected cases.

effectiveR(postProcessor::PostProcessor)

Returns a DataFrame containing the effective R value for each tick.

For each infectee, this method looks ahead for secondary infections this individual might cause during the total span of the simulation. These infections are then counted towards the R-value of the initial infection. If individual A, for example, is infected at time 42 and causes four secondary infections during the next 14 ticks, these four infections are counted towards the R-value of time 42.

Note: This only works in scenarios without re-infection as the current implementation just evaluates the total infections caused by each individual in general. If an individual was infected multiple times, secondary infections will inflate the statistic.

Returns

• Dataframe with the following columns:

hospital_df(postProcessor::PostProcessor)

Creates a DataFrame that includes information about the current hospitalizations etc.

Returns

• DataFrame with the following columns:

household_attack_rates(postProcessor::PostProcessor; hh_samples::Int64 = HOUSEHOLD_ATTACK_RATE_SAMPLES)

Returns a DataFrame containing data on the in-household attack rate. The in-household attack rate is defined as the fraction of individuals in a given household that got infected within the household (in-household infection chain) caused by the first introduction of the pathogen in this household. It does not reflect overall fraction of individuals that were infected in this household throughout the course of the simuation. As the attack rate calculation is very computationally intensive, it is not done for all household but rather for a subset of households. You can change the desired subset size through the optional hh_samples argument. Its default can be found in constants.jl

Returns

• DataFrame with the following columns:

individuals_per_age_group(post_processor::PostProcessor, interval_steps::Int64)

The population in populationDF will be splitted in age groups of size interval_steps and the number of individuals per age group is counted.

The input DataFrame already needs a column called "sum" where the sum of individuals in this age is stored.

Returns

• returns a DataFrame with the follwing structure:

individuals_per_age_group(post_processor::PostProcessor, interval_steps::Int64, aggregation_bound::Int64)

The the population in populationDF will be splitted in age groups of size interval_steps and the number of individuals per age group is counted. The splitting is capped at aggregation_bound.

The input DataFrame already needs a column called "sum" where the sum of individuals in this age is stored.

Returns

• returns a DataFrame with the follwing structure:

infections(postProcessor::PostProcessor)

Returns the internal flat infections DataFrame.

Columns

infectionsDF(postProcessor::PostProcessor)

Returns the internal flat infections DataFrame. Lookup the docstring of infections(postProcessor::PostProcessor) for column definitions.

observed_R(postProcessor::PostProcessor)

Returns a DataFrame containing estimations for the effective (current) reproduction number R, based on detected infections. The calculation uses the estimation for the (rolling) observed serial interval (SI) and the total of detected cases within a time window of the length defined in R_ESTIMATION_TIME_WINDOW (lookup constants.jl) ending with a current tick t. It divides the total detected infections in the current time window with the total detected infections in the time window that lies one SI behind. The function does this for the mean estimate of the serial interval as well as the (floored) lower-95% confidence bound and the (ceiled) upper-95% confidence bound.

Example

• For tick t = 42 and
• Given the R_ESTIMATION_TIME_WINDOW = 7, and
• The rolling observed serial interval being 4,
• This function does: sum(infections[35:42]) / sum(infections[31:38])

Returns

• DataFrame with the following columns:

pooltestsDF(postProcessor::PostProcessor)

Returns the internal flat pool tests DataFrame.

Columns

populationDF(postProcessor)

Returns the internal flat population DataFrame.

Columns

population_pyramid(postProcessor::PostProcessor)

Returns a DataFrame containing data to generate a population pyramid. It provides the sum of all female and male individuals in all age groups in the population model. Sums for female are multiplied by -1 to facilitate visualization.

Returns

• DataFrame with the following columns:

reported_tick_cases(postProcessor::PostProcessor)

Returns the number of reported positive test cases per tick. This analysis is based on the testDF dataframe.

Returns

• DataFrame with the following columns:

rolling_observed_SI(postProcessor::PostProcessor)

Returns a DataFrame containing aggregated estimations on the serial interval based on true detected cases. The estimations are based on all true detected cases in a 14-past-days time window. If fewer than 50 infections were recorded in that time window, detections prior to that are added until the sample is complete. This is done in order to reduce large stochastic fluctuations due to small sample size.

Returns

• DataFrame with the following columns:

setting_sizes(postProcessor::PostProcessor)

Returns a Dictionary containing information about size of the settings. The keys are equal to the settingtypes and the values correspond to a countmap of the setting sizes.

Returns

• Dict{String, Dict{Int64, Int64}}: Nested dictionary where the first key is the name of the setting type (e.g., "Household") and the innter dictionary is a countmap with the key being a setting size (e.g., 5) and the value the number of occurences.

settingdata(postProcessor::PostProcessor)

Returns a DataFrame containing information about setting types

Returns

• DataFrame with the following columns:

sim_infectionsDF(postProcessor::PostProcessor)

Returns a DataFrame containing all infections that happened during the simulation run. As it is a direct filter on the PostProcessors internal infectionsDF, the column structure is identical to the output of infectionsDF(postProcessor)

simulation(postProcessor::PostProcessor)

Returns the associated Simulation object.

testsDF(postProcessor::PostProcessor)

Returns the internal flat tests DataFrame. It was joined with the population dataframe to also obtain personal characteristics about the testees.

Columns

tick_cases(postProcessor::PostProcessor)

Returns a DataFrame containing the count of individuals currently entering in the respective disease states exposed, infectious, and removed.

Returns

• DataFrame with the following columns:

tick_cases_per_setting(postProcessor::PostProcessor)

Returns a DataFrame containing information about the infections in different setting types

Returns

• DataFrame with the following columns:

tick_deaths(postProcessor::PostProcessor)

Returns a DataFrame containing the count of individuals that died per tick.

Returns

• DataFrame with the following columns:

tick_generation_times(postProcessor::PostProcessor)

Returns a DataFrame containing aggregated information on the generation time per tick.

Returns

• DataFrame with the following columns:

 tick_pooltests(postProcessor::PostProcessor)

Returns a Dict for each employed test_type containing their name as a key and a DataFrame containing the number of applied pool tests (positive/negative/total) per tick.

Returns

• Dict{String, DataFrame}: The key is the TestType's name. The values are DataFrames with the following columns:

tick_serial_intervals(postProcessor::PostProcessor)

Returns a DataFrame containing aggregated information on the serial interval per tick.

Returns

• DataFrame with the follwing columns:

tick_tests(postProcessor::PostProcessor)

Returns a Dict for each employed test_type containing their name as a key and a DataFrame containing the count of individuals that got tested (positive/negative) as well as the people reported positive for the first time per tick.

Returns

• Dict{String, DataFrame}: The key is the TestType's name. The values are DataFrames with the following columns:

time_to_detection(postProcessor::PostProcessor)

Returns the mean, standard deviation, minimum, maximum, upper- and lower 95% confidence intervals of the time to detection for all detected cases. The time to detection is defined as the number of ticks between the time of exposure and time of detection.

Returns

• DataFrame with the following columns:

total_detected_cases(postProcessor::PostProcessor)

Returns the total number of detected cases.

total_quarantines(postProcessor::PostProcessor)

Returns the total number of total ticks spent in quarantine. E.g., if 10 indiviuals were in quarantine for 5 ticks each, this function will return 50.

total_tests(postProcessor::PostProcessor)

Sums up the total number of tests per test type.

Returns

• Dict{String, Int64}: Dictionary where the key is the TestType's name and the value the number of tests that were applied.

weighted_error_sum(post_processor::PostProcessor, error_matrix::ContactMatrix{T})::T where T <: Number

Calculate a weighted error sum for the given contact matrix. Each cell will be multiplied by the number of individuals in the corresponding age group and then divided by the number of individuals in the whole population.

weighted_error_sum = (cell value * individuals per age group) / population size)

To calculate the weighted sum, "weighted arithmetic mean" is used.

Assumptions

• error_matrix Matrix contains error values between two contact matrices.

weighted_error_sum(post_processor::PostProcessor, setting::DataType, reference_matrix::ContactMatrix{T}; fit_to_reference_matrix::Bool)::T where T <: Number

Calculate a "weighted error sum" for a setting specific contact matrix compared to a reference_matrix. The post_processor is used to calculate a contact matrix for the associated simulation that will then be fitted by a factor alpha calculated from the differences between the simulation contact matrix and the reference_matrix.

Compared to weighted_error_sum(post_processor::PostProcessor, error_matrix::ContactMatrix{T})::T where T <: Number the error_matrix first needs to be calculated using the post_processor and the reference_matrix.

Each cell of the calculated error matrix will be multiplied by the number of individuals in the corresponding age group and then divided by the number of individuals in the whole population of the simulation.

weighted_error_sum = (cell value * individuals per age group) / population size)

To calculate the weighted sum, "weighted arithmetic mean" is used.

Parameters

• post_processor::PostProcessor: PostProcessor object containing the simulation and population data
• setting::DataType: Settingtype of the reference_matrix (i.e. Household)
• reference_matrix::ContactMatrix{T}: Matrix containing mean number of contacts per age group between individuals of one settingtype.
• fit_to_reference_matrix::Bool: Flag, indicating if the matrix calculated from the simulation should be fitted to the 'reference_matrix`.

Assumptions

• reference_matrix logically stems from the same setting as setting!