Background

The dplyr package provides a language, or grammar, for data manipulation.

The language contains a number of verbs that operate on tables in data frames.

The most commonly used verbs operate on a single data frame:

There are also a number of join verbs that merge several data frames into one.

The tidyr package provides additional verbs, such as pivot_longer and pivot_wider for reshaping data frames.

The single table verbs can also be used with group_by() and ungroup() to work a group at a time instead of applying to the entire data frame.

The design of dplyr is strongly motivated by SQL.

Some data sets for illustration:

library(readr)
if (! file.exists("vehicles.csv.zip"))
    download.file("http://www.stat.uiowa.edu/~luke/data/vehicles.csv.zip",
                  "vehicles.csv.zip")
newmpg <- read_csv("vehicles.csv.zip", guess_max = 100000)
library(nycflights13)

The basic transformation verbs are described in the Data Transformation chapter of R for Data Science.

Utilities in tidyr are described in the Tidy Data chapter.

Selecting Variables

Data sets often contain hundreds of even thousands of variables.

A useful first step is to select a group of interesting variables.

A reasonable selection of the EPA vehicle variables:

newmpg1 <- select(newmpg,
                  make, model, year,
                  cty = city08, hwy = highway08,
                  trans = trany, cyl = cylinders,
                  fuel = fuelType1, displ)
head(newmpg1, 2)
## # A tibble: 2 × 9
##   make       model               year   cty   hwy trans          cyl fuel  displ
##   <chr>      <chr>              <dbl> <dbl> <dbl> <chr>        <dbl> <chr> <dbl>
## 1 Alfa Romeo Spider Veloce 2000  1985    19    25 Manual 5-spd     4 Regu…   2  
## 2 Ferrari    Testarossa          1985     9    14 Manual 5-spd    12 Regu…   4.9

Variables can be given new names in a select call; you can also rename them later with rename.

Some variations (the documentation for select() gives full details):

select(newmpg1, year : trans) %>% head(2)
## # A tibble: 2 × 4
##    year   cty   hwy trans       
##   <dbl> <dbl> <dbl> <chr>       
## 1  1985    19    25 Manual 5-spd
## 2  1985     9    14 Manual 5-spd
select(newmpg1, -year, -trans) %>% head(2)
## # A tibble: 2 × 7
##   make       model                cty   hwy   cyl fuel             displ
##   <chr>      <chr>              <dbl> <dbl> <dbl> <chr>            <dbl>
## 1 Alfa Romeo Spider Veloce 2000    19    25     4 Regular Gasoline   2  
## 2 Ferrari    Testarossa             9    14    12 Regular Gasoline   4.9
select(newmpg1, - (year : trans)) %>% head(2)
## # A tibble: 2 × 5
##   make       model                cyl fuel             displ
##   <chr>      <chr>              <dbl> <chr>            <dbl>
## 1 Alfa Romeo Spider Veloce 2000     4 Regular Gasoline   2  
## 2 Ferrari    Testarossa            12 Regular Gasoline   4.9

Numerical column specifications can also be used:

select(newmpg1, 1, 3) %>% head(2)
## # A tibble: 2 × 2
##   make        year
##   <chr>      <dbl>
## 1 Alfa Romeo  1985
## 2 Ferrari     1985
select(newmpg1, 1 : 3) %>% head(2)
## # A tibble: 2 × 3
##   make       model               year
##   <chr>      <chr>              <dbl>
## 1 Alfa Romeo Spider Veloce 2000  1985
## 2 Ferrari    Testarossa          1985
select(newmpg1, - (1 : 3)) %>% head(2)
## # A tibble: 2 × 6
##     cty   hwy trans          cyl fuel             displ
##   <dbl> <dbl> <chr>        <dbl> <chr>            <dbl>
## 1    19    25 Manual 5-spd     4 Regular Gasoline   2  
## 2     9    14 Manual 5-spd    12 Regular Gasoline   4.9

Some helper functions can also be used in the column specifications.

The most useful ones are starts_with, ends_with, and contains.

select(newmpg, starts_with("fuel")) %>% head(2)
## # A tibble: 2 × 5
##   fuelCost08 fuelCostA08 fuelType fuelType1        fuelType2
##        <dbl>       <dbl> <chr>    <chr>            <chr>    
## 1       2450           0 Regular  Regular Gasoline <NA>     
## 2       4650           0 Regular  Regular Gasoline <NA>
select(newmpg, contains("city")) %>% head(2)
## # A tibble: 2 × 12
##   city08 city08U cityA08 cityA08U cityCD cityE cityUF rangeCity rangeCityA UCity
##    <dbl>   <dbl>   <dbl>    <dbl>  <dbl> <dbl>  <dbl>     <dbl>      <dbl> <dbl>
## 1     19       0       0        0      0     0      0         0          0  23.3
## 2      9       0       0        0      0     0      0         0          0  11  
## # ℹ 2 more variables: UCityA <dbl>, phevCity <dbl>

By default these helpers ignore case.

These can also be preceded by a minus modifier to omit columns:

select(newmpg1, -contains("m")) %>% head(2)
## # A tibble: 2 × 7
##    year   cty   hwy trans          cyl fuel             displ
##   <dbl> <dbl> <dbl> <chr>        <dbl> <chr>            <dbl>
## 1  1985    19    25 Manual 5-spd     4 Regular Gasoline   2  
## 2  1985     9    14 Manual 5-spd    12 Regular Gasoline   4.9

contains requires a literal string; matches is analogous but its argument is interpreted as a regular expression pattern to be matched.

select(newmpg, matches("^[Ff]uel")) %>% head(2)
## # A tibble: 2 × 5
##   fuelCost08 fuelCostA08 fuelType fuelType1        fuelType2
##        <dbl>       <dbl> <chr>    <chr>            <chr>    
## 1       2450           0 Regular  Regular Gasoline <NA>     
## 2       4650           0 Regular  Regular Gasoline <NA>

Sometimes it is useful to move a few columns to the front; the everything helper can be used for that.

The original NYC flights table:

head(flights, 2)
## # A tibble: 2 × 19
##    year month   day dep_time sched_dep_time dep_delay arr_time sched_arr_time
##   <int> <int> <int>    <int>          <int>     <dbl>    <int>          <int>
## 1  2013     1     1      517            515         2      830            819
## 2  2013     1     1      533            529         4      850            830
## # ℹ 11 more variables: arr_delay <dbl>, carrier <chr>, flight <int>,
## #   tailnum <chr>, origin <chr>, dest <chr>, air_time <dbl>, distance <dbl>,
## #   hour <dbl>, minute <dbl>, time_hour <dttm>

Moving air_time to the front:

select(flights, air_time, everything()) %>% head(2)
## # A tibble: 2 × 19
##   air_time  year month   day dep_time sched_dep_time dep_delay arr_time
##      <dbl> <int> <int> <int>    <int>          <int>     <dbl>    <int>
## 1      227  2013     1     1      517            515         2      830
## 2      227  2013     1     1      533            529         4      850
## # ℹ 11 more variables: sched_arr_time <int>, arr_delay <dbl>, carrier <chr>,
## #   flight <int>, tailnum <chr>, origin <chr>, dest <chr>, distance <dbl>,
## #   hour <dbl>, minute <dbl>, time_hour <dttm>

The where helper function allows columns to be selected based on a predicate applied to the full column:

select(newmpg1, where(anyNA)) %>% head(2)
## # A tibble: 2 × 3
##   trans          cyl displ
##   <chr>        <dbl> <dbl>
## 1 Manual 5-spd     4   2  
## 2 Manual 5-spd    12   4.9
select(flights, where(anyNA)) %>% head(2)
## # A tibble: 2 × 6
##   dep_time dep_delay arr_time arr_delay tailnum air_time
##      <int>     <dbl>    <int>     <dbl> <chr>      <dbl>
## 1      517         2      830        11 N14228       227
## 2      533         4      850        20 N24211       227

If there are missing values in your data it is usually a good idea to review them to make sure you understand what they mean.

Columns can also be selected using list operations:

vars <- c("make", "model", "year", "city08", "trany", "cylinders",
          "fuelType1", "displ")
newmpg[vars] %>% head(2)
## # A tibble: 2 × 8
##   make       model               year city08 trany     cylinders fuelType1 displ
##   <chr>      <chr>              <dbl>  <dbl> <chr>         <dbl> <chr>     <dbl>
## 1 Alfa Romeo Spider Veloce 2000  1985     19 Manual 5…         4 Regular …   2  
## 2 Ferrari    Testarossa          1985      9 Manual 5…        12 Regular …   4.9

Filtering Rows

The filter function picks out the subset of rows that satisfy one or more conditions.

All cars with city MPG at or above 130 and model year 2018:

filter(newmpg1, cty >= 130, year == 2018)
## # A tibble: 2 × 9
##   make    model               year   cty   hwy trans            cyl fuel   displ
##   <chr>   <chr>              <dbl> <dbl> <dbl> <chr>          <dbl> <chr>  <dbl>
## 1 Hyundai Ioniq Electric      2018   150   122 Automatic (A1)    NA Elect…    NA
## 2 Tesla   Model 3 Long Range  2018   136   123 Automatic (A1)    NA Elect…    NA

All flights from New York to Des Moines on January 1, 2013:

filter(flights, day == 1, month == 1, dest == "DSM") %>%
    select(year : dep_time, origin)
## # A tibble: 1 × 5
##    year month   day dep_time origin
##   <int> <int> <int>    <int> <chr> 
## 1  2013     1     1     2119 EWR

Comparisons

The basic comparison operators are

  • ==, !=
  • <, <=
  • >, >=

Be sure to use ==, not a single = character.

Be careful about floating point equality tests:

x <- 1 - 0.9
x == 0.1
## [1] FALSE
near(x, 0.1)
## [1] TRUE

Equality comparisons can be used on character vectors and factors.

Order comparisons can be used on character vectors, but may produce surprising results, or results that vary with different locale settings.

%in% can be used to match against one of several options:

filter(flights, day %in% 1 : 2, month == 1, dest == "DSM") %>%
    select(year : dep_time, origin)
## # A tibble: 2 × 5
##    year month   day dep_time origin
##   <int> <int> <int>    <int> <chr> 
## 1  2013     1     1     2119 EWR   
## 2  2013     1     2     1921 EWR

Logical Operations

The basic logical operations on vectors are

  • & – and
  • | – or
  • ! – not
  • xor – exclusive or

Truth tables:

x <- c(TRUE, TRUE,  FALSE, FALSE)
y <- c(TRUE, FALSE, TRUE,  FALSE)
! x
## [1] FALSE FALSE  TRUE  TRUE
x & y
## [1]  TRUE FALSE FALSE FALSE
x | y
## [1]  TRUE  TRUE  TRUE FALSE
xor(x, y)
## [1] FALSE  TRUE  TRUE FALSE

Make sure not to confuse the vectorized logical operators & and | with the scalar flow control operators && and ||.

The previous flights example can also be written as

filter(flights, day == 1 | day == 2, month == 1, dest == "DSM") %>%
    select(year : dep_time, origin)
## # A tibble: 2 × 5
##    year month   day dep_time origin
##   <int> <int> <int>    <int> <chr> 
## 1  2013     1     1     2119 EWR   
## 2  2013     1     2     1921 EWR

It can also be written as

filter(flights, (day == 1 | day == 2) & month == 1 & dest == "DSM") %>%
    select(year : dep_time, origin)
## # A tibble: 2 × 5
##    year month   day dep_time origin
##   <int> <int> <int>    <int> <chr> 
## 1  2013     1     1     2119 EWR   
## 2  2013     1     2     1921 EWR

Missing Values

filter only keeps values corresponding to TRUE predicate values; FALSE and NA values are dropped.

Arithmetic and comparison operators will always produce NA values if one operand is NA.

1 + NA
## [1] NA
NA < 1
## [1] NA
NA == NA
## [1] NA

In a few cases a non-NA result can be determined:

TRUE | NA
## [1] TRUE
NA & FALSE
## [1] FALSE
NA ^ 0
## [1] 1

is.na can be used to also select rows with NA values.

Non-electric vehicles with zero or NA for displ:

filter(newmpg1, displ == 0 | is.na(displ), fuel != "Electricity")
## # A tibble: 2 × 9
##   make   model     year   cty   hwy trans          cyl fuel             displ
##   <chr>  <chr>    <dbl> <dbl> <dbl> <chr>        <dbl> <chr>            <dbl>
## 1 Subaru RX Turbo  1985    22    28 Manual 5-spd    NA Regular Gasoline    NA
## 2 Subaru RX Turbo  1985    21    27 Manual 5-spd    NA Regular Gasoline    NA

Flights with NA for both dep_time and arr_time:

filter(flights, is.na(dep_time) & is.na(arr_time)) %>%
    head(2) %>%
    select(year : arr_time)
## # A tibble: 2 × 7
##    year month   day dep_time sched_dep_time dep_delay arr_time
##   <int> <int> <int>    <int>          <int>     <dbl>    <int>
## 1  2013     1     1       NA           1630        NA       NA
## 2  2013     1     1       NA           1935        NA       NA

An alternative approach that does not use filter would be

idx <- with(flights, is.na(dep_time) & is.na(arr_time))
flights[idx, ] %>%
    head(2) %>%
    select(year : arr_time)
## # A tibble: 2 × 7
##    year month   day dep_time sched_dep_time dep_delay arr_time
##   <int> <int> <int>    <int>          <int>     <dbl>    <int>
## 1  2013     1     1       NA           1630        NA       NA
## 2  2013     1     1       NA           1935        NA       NA

Exploring the EPA Data

Sorting out fuel types:

select(newmpg, contains("fuelType")) %>% unique()
## # A tibble: 14 × 3
##    fuelType                    fuelType1         fuelType2  
##    <chr>                       <chr>             <chr>      
##  1 Regular                     Regular Gasoline  <NA>       
##  2 Premium                     Premium Gasoline  <NA>       
##  3 Diesel                      Diesel            <NA>       
##  4 CNG                         Natural Gas       <NA>       
##  5 Gasoline or natural gas     Regular Gasoline  Natural Gas
##  6 Gasoline or E85             Regular Gasoline  E85        
##  7 Electricity                 Electricity       <NA>       
##  8 Gasoline or propane         Regular Gasoline  Propane    
##  9 Premium or E85              Premium Gasoline  E85        
## 10 Midgrade                    Midgrade Gasoline <NA>       
## 11 Premium Gas or Electricity  Premium Gasoline  Electricity
## 12 Regular Gas and Electricity Regular Gasoline  Electricity
## 13 Premium and Electricity     Premium Gasoline  Electricity
## 14 Regular Gas or Electricity  Regular Gasoline  Electricity

Variables with missing values:

select(newmpg, where(anyNA)) %>% head(2)
## # A tibble: 2 × 18
##   cylinders displ drive       eng_dscr mpgData trany guzzler trans_dscr tCharger
##       <dbl> <dbl> <chr>       <chr>    <chr>   <chr> <chr>   <chr>      <lgl>   
## 1         4   2   Rear-Wheel… (FFS)    Y       Manu… <NA>    <NA>       NA      
## 2        12   4.9 Rear-Wheel… (GUZZLE… N       Manu… T       <NA>       NA      
## # ℹ 9 more variables: sCharger <chr>, atvType <chr>, fuelType2 <chr>,
## #   rangeA <chr>, evMotor <chr>, mfrCode <chr>, c240Dscr <chr>,
## #   c240bDscr <chr>, startStop <chr>

Among the reduced data set:

select(newmpg1, where(anyNA)) %>% names()
## [1] "trans" "cyl"   "displ"

Unique missing value patterns:

incomplete_cases <- function(data) data[! complete.cases(data), ]

select(newmpg1, trans, cyl, displ, fuel) %>%
    incomplete_cases() %>%
    unique()
## # A tibble: 9 × 4
##   trans                              cyl displ fuel            
##   <chr>                            <dbl> <dbl> <chr>           
## 1 <NA>                                NA  NA   Electricity     
## 2 <NA>                                 8   5.8 Regular Gasoline
## 3 <NA>                                 6   4.1 Regular Gasoline
## 4 Manual 5-spd                        NA  NA   Regular Gasoline
## 5 Manual 5-spd                        NA   1.3 Regular Gasoline
## 6 Automatic (A1)                      NA  NA   Electricity     
## 7 Automatic (variable gear ratios)    NA  NA   Electricity     
## 8 Automatic (A1)                      NA   0   Electricity     
## 9 Automatic (A2)                      NA  NA   Electricity

An alternative to defining incomplete_cases():

select(newmpg1, trans, cyl, displ, fuel) %>%
    filter(if_any(everything(), is.na)) %>%
    unique()
## # A tibble: 9 × 4
##   trans                              cyl displ fuel            
##   <chr>                            <dbl> <dbl> <chr>           
## 1 <NA>                                NA  NA   Electricity     
## 2 <NA>                                 8   5.8 Regular Gasoline
## 3 <NA>                                 6   4.1 Regular Gasoline
## 4 Manual 5-spd                        NA  NA   Regular Gasoline
## 5 Manual 5-spd                        NA   1.3 Regular Gasoline
## 6 Automatic (A1)                      NA  NA   Electricity     
## 7 Automatic (variable gear ratios)    NA  NA   Electricity     
## 8 Automatic (A1)                      NA   0   Electricity     
## 9 Automatic (A2)                      NA  NA   Electricity

Looking at the counts:

select(newmpg1, trans, cyl, displ, fuel) %>%
    incomplete_cases() %>%
    count(trans, cyl, displ, fuel)
## # A tibble: 9 × 5
##   trans                              cyl displ fuel                 n
##   <chr>                            <dbl> <dbl> <chr>            <int>
## 1 Automatic (A1)                      NA   0   Electricity          1
## 2 Automatic (A1)                      NA  NA   Electricity        422
## 3 Automatic (A2)                      NA  NA   Electricity         43
## 4 Automatic (variable gear ratios)    NA  NA   Electricity          9
## 5 Manual 5-spd                        NA   1.3 Regular Gasoline     1
## 6 Manual 5-spd                        NA  NA   Regular Gasoline     2
## 7 <NA>                                 6   4.1 Regular Gasoline     1
## 8 <NA>                                 8   5.8 Regular Gasoline     1
## 9 <NA>                                NA  NA   Electricity          9

count(trans, cyl, displ, fuel) is essentially an abbreviation of

group_by(trans, cyl, displ, fuel) %>%
    summarize(n = n()) %>%
    ungroup()

Another approach that avoids specifying the variables with missing values:

incomplete_cases(newmpg1) %>%
    count(across(where(anyNA)), fuel)
## # A tibble: 9 × 5
##   trans                              cyl displ fuel                 n
##   <chr>                            <dbl> <dbl> <chr>            <int>
## 1 Automatic (A1)                      NA   0   Electricity          1
## 2 Automatic (A1)                      NA  NA   Electricity        422
## 3 Automatic (A2)                      NA  NA   Electricity         43
## 4 Automatic (variable gear ratios)    NA  NA   Electricity          9
## 5 Manual 5-spd                        NA   1.3 Regular Gasoline     1
## 6 Manual 5-spd                        NA  NA   Regular Gasoline     2
## 7 <NA>                                 6   4.1 Regular Gasoline     1
## 8 <NA>                                 8   5.8 Regular Gasoline     1
## 9 <NA>                                NA  NA   Electricity          9

Cylinders for electric vehicles:

filter(newmpg1, fuel == "Electricity") %>%
    count(is.na(cyl))
## # A tibble: 1 × 2
##   `is.na(cyl)`     n
##   <lgl>        <int>
## 1 TRUE           484

Displacement for electric vehicles:

filter(newmpg1, fuel == "Electricity") %>%
    count(is.na(displ))
## # A tibble: 2 × 2
##   `is.na(displ)`     n
##   <lgl>          <int>
## 1 FALSE              1
## 2 TRUE             483

Electric vehicles with non-missing displacement:

filter(newmpg1, fuel == "Electricity", ! is.na(displ))
## # A tibble: 1 × 9
##   make       model   year   cty   hwy trans            cyl fuel        displ
##   <chr>      <chr>  <dbl> <dbl> <dbl> <chr>          <dbl> <chr>       <dbl>
## 1 Mitsubishi i-MiEV  2016   126    99 Automatic (A1)    NA Electricity     0

Some things to think about:

Adding New Variables

mutate() can be used to define new variables or modify existing ones.

New variables are added at the end.

Later variables can be defined in terms of earlier ones.

mutate() does not modify the original data frame; it creates a new one with the specified modifications.

fl <- select(flights, year, month, day, air_time, ends_with("delay"))
mutate(fl,
       gain = dep_delay - arr_delay,
       air_hours = air_time / 60,
       gain_per_hour = gain / air_hours) %>%
    head(2)
## # A tibble: 2 × 9
##    year month   day air_time dep_delay arr_delay  gain air_hours gain_per_hour
##   <int> <int> <int>    <dbl>     <dbl>     <dbl> <dbl>     <dbl>         <dbl>
## 1  2013     1     1      227         2        11    -9      3.78         -2.38
## 2  2013     1     1      227         4        20   -16      3.78         -4.23

transmute() keeps only the new variables.

transmute(flights,
          gain = dep_delay - arr_delay,
          air_hours = air_time / 60,
          gain_per_hour = gain / air_hours) %>%
    head(2)
## # A tibble: 2 × 3
##    gain air_hours gain_per_hour
##   <dbl>     <dbl>         <dbl>
## 1    -9      3.78         -2.38
## 2   -16      3.78         -4.23

Useful operators include:

A reasonable guess is that flights with NA values for both dep_time and arr_time were cancelled:

fl <- mutate(flights,
             cancelled = is.na(dep_time) & is.na(arr_time))

For the geyser data set from the MASS package we used lag to obtain the duration of the previous eruption,

data(geyser, package = "MASS")
geyser2 <- mutate(geyser,
                  prev_duration = lag(duration))
head(geyser2, 4)
##   waiting duration prev_duration
## 1      80 4.016667            NA
## 2      71 2.150000      4.016667
## 3      57 4.000000      2.150000
## 4      80 4.000000      4.000000
thm <- theme_minimal() +
    theme(text = element_text(size = 16))
ggplot(geyser2) +
    geom_point(aes(x = prev_duration,
                   y = waiting)) +
    thm
## Warning: Removed 1 rows containing missing values (`geom_point()`).

The top five vehicles in cty gas mileage (there may be more than five if there is a tie for fifth place):

mutate(newmpg1, cty_rank = min_rank(desc(cty))) %>%
    filter(cty_rank <= 5)
## # A tibble: 5 × 10
##   make    model                year   cty   hwy trans   cyl fuel  displ cty_rank
##   <chr>   <chr>               <dbl> <dbl> <dbl> <chr> <dbl> <chr> <dbl>    <int>
## 1 Hyundai Ioniq Electric       2017   150   122 Auto…    NA Elec…    NA        1
## 2 Hyundai Ioniq Electric       2018   150   122 Auto…    NA Elec…    NA        1
## 3 Hyundai Ioniq Electric       2019   150   122 Auto…    NA Elec…    NA        1
## 4 Tesla   Model 3 Standard R…  2020   148   132 Auto…    NA Elec…    NA        5
## 5 Tesla   Model 3 Standard R…  2021   150   133 Auto…    NA Elec…    NA        1

An alternative to explicitly computing ranks and filtering is the slice_max() utility function:

slice_max(newmpg1, cty, n = 5)
## # A tibble: 5 × 9
##   make    model                         year   cty   hwy trans   cyl fuel  displ
##   <chr>   <chr>                        <dbl> <dbl> <dbl> <chr> <dbl> <chr> <dbl>
## 1 Hyundai Ioniq Electric                2017   150   122 Auto…    NA Elec…    NA
## 2 Hyundai Ioniq Electric                2018   150   122 Auto…    NA Elec…    NA
## 3 Hyundai Ioniq Electric                2019   150   122 Auto…    NA Elec…    NA
## 4 Tesla   Model 3 Standard Range Plus…  2021   150   133 Auto…    NA Elec…    NA
## 5 Tesla   Model 3 Standard Range Plus   2020   148   132 Auto…    NA Elec…    NA

The percent_rank function produces a ‘percentile rank’ (but between 0 and 1).

The top 5% of countries in life expectancy in 2007 from the gapminder data set:

library(gapminder)
gm <- filter(gapminder, year == 2007)
gm_top <- filter(gm, percent_rank(desc(lifeExp)) <= 0.05)
gm_top
## # A tibble: 8 × 6
##   country          continent  year lifeExp       pop gdpPercap
##   <fct>            <fct>     <int>   <dbl>     <int>     <dbl>
## 1 Australia        Oceania    2007    81.2  20434176    34435.
## 2 Hong Kong, China Asia       2007    82.2   6980412    39725.
## 3 Iceland          Europe     2007    81.8    301931    36181.
## 4 Israel           Asia       2007    80.7   6426679    25523.
## 5 Japan            Asia       2007    82.6 127467972    31656.
## 6 Spain            Europe     2007    80.9  40448191    28821.
## 7 Sweden           Europe     2007    80.9   9031088    33860.
## 8 Switzerland      Europe     2007    81.7   7554661    37506.

And the bottom 5%:

filter(gm, percent_rank(lifeExp) <= 0.05)
## # A tibble: 8 × 6
##   country      continent  year lifeExp      pop gdpPercap
##   <fct>        <fct>     <int>   <dbl>    <int>     <dbl>
## 1 Afghanistan  Asia       2007    43.8 31889923      975.
## 2 Angola       Africa     2007    42.7 12420476     4797.
## 3 Lesotho      Africa     2007    42.6  2012649     1569.
## 4 Mozambique   Africa     2007    42.1 19951656      824.
## 5 Sierra Leone Africa     2007    42.6  6144562      863.
## 6 Swaziland    Africa     2007    39.6  1133066     4513.
## 7 Zambia       Africa     2007    42.4 11746035     1271.
## 8 Zimbabwe     Africa     2007    43.5 12311143      470.

Within these results the original row order is retained.

The arrange function can be used to change this.

Arranging Rows

arrange() reorders the rows according to one or more variables.

By default the order is smallest to largest; using desc() reverses this.

Additional variables will be used to break ties.

Arranging the top 5% of countries in life expectancy in 2007 by life expectancy:

arrange(gm_top, desc(lifeExp))
## # A tibble: 8 × 6
##   country          continent  year lifeExp       pop gdpPercap
##   <fct>            <fct>     <int>   <dbl>     <int>     <dbl>
## 1 Japan            Asia       2007    82.6 127467972    31656.
## 2 Hong Kong, China Asia       2007    82.2   6980412    39725.
## 3 Iceland          Europe     2007    81.8    301931    36181.
## 4 Switzerland      Europe     2007    81.7   7554661    37506.
## 5 Australia        Oceania    2007    81.2  20434176    34435.
## 6 Spain            Europe     2007    80.9  40448191    28821.
## 7 Sweden           Europe     2007    80.9   9031088    33860.
## 8 Israel           Asia       2007    80.7   6426679    25523.

To order by continent name and order by life expectancy within continent:

arrange(gm_top, continent, desc(lifeExp))
## # A tibble: 8 × 6
##   country          continent  year lifeExp       pop gdpPercap
##   <fct>            <fct>     <int>   <dbl>     <int>     <dbl>
## 1 Japan            Asia       2007    82.6 127467972    31656.
## 2 Hong Kong, China Asia       2007    82.2   6980412    39725.
## 3 Israel           Asia       2007    80.7   6426679    25523.
## 4 Iceland          Europe     2007    81.8    301931    36181.
## 5 Switzerland      Europe     2007    81.7   7554661    37506.
## 6 Spain            Europe     2007    80.9  40448191    28821.
## 7 Sweden           Europe     2007    80.9   9031088    33860.
## 8 Australia        Oceania    2007    81.2  20434176    34435.

Missing values are sorted to the end:

df <- data.frame(x = c(5, 2, NA, 3))
arrange(df, x)
##    x
## 1  2
## 2  3
## 3  5
## 4 NA
arrange(df, desc(x))
##    x
## 1  5
## 2  3
## 3  2
## 4 NA

Using is.na you can arrange to put them at the beginning:

arrange(df, desc(is.na(x)), x)
##    x
## 1 NA
## 2  2
## 3  3
## 4  5

Summarizing and Grouping

summarize() collapses a table down to one row of summaries.

The average and maximal departure and arrival delays for flights, along with the total number of flights:

summarize(flights,
          ave_dep_delay = mean(dep_delay, na.rm = TRUE),
          ave_arr_delay = mean(arr_delay, na.rm = TRUE),
          max_dep_delay = max(dep_delay, na.rm = TRUE),
          max_arr_delay = max(arr_delay, na.rm = TRUE),
          n = n())
## # A tibble: 1 × 5
##   ave_dep_delay ave_arr_delay max_dep_delay max_arr_delay      n
##           <dbl>         <dbl>         <dbl>         <dbl>  <int>
## 1          12.6          6.90          1301          1272 336776

Grouped Summaries

summarize() is most commonly used with grouped data to provide group level summaries.

Grouping delay summaries by destination:

fl_dest <-
    group_by(flights, dest) %>%
    summarize(ave_dep_delay = mean(dep_delay, na.rm = TRUE),
              ave_arr_delay = mean(arr_delay, na.rm = TRUE),
              max_dep_delay = max(dep_delay, na.rm = TRUE),
              max_arr_delay = max(arr_delay, na.rm = TRUE),
              n = n()) %>%
    ungroup()
head(fl_dest, 5)
## # A tibble: 5 × 6
##   dest  ave_dep_delay ave_arr_delay max_dep_delay max_arr_delay     n
##   <chr>         <dbl>         <dbl>         <dbl>         <dbl> <int>
## 1 ABQ           13.7           4.38           142           153   254
## 2 ACK            6.46          4.85           219           221   265
## 3 ALB           23.6          14.4            323           328   439
## 4 ANC           12.9          -2.5             75            39     8
## 5 ATL           12.5          11.3            898           895 17215

Calling ungroup is not always necessary but sometimes it is, so it is a good habit to get into.

The group level summaries can be further transformed, for example to identify the top 10 destinations for average delays:

mutate(fl_dest, rank = min_rank(desc(ave_dep_delay))) %>%
    filter(rank <= 10)
## # A tibble: 10 × 7
##    dest  ave_dep_delay ave_arr_delay max_dep_delay max_arr_delay     n  rank
##    <chr>         <dbl>         <dbl>         <dbl>         <dbl> <int> <int>
##  1 ALB            23.6          14.4           323           328   439     9
##  2 BHM            29.7          16.9           325           291   297     4
##  3 CAE            35.6          41.8           222           224   116     1
##  4 DSM            26.2          19.0           341           322   569     7
##  5 JAC            26.5          28.1           198           175    25     6
##  6 MSN            23.6          20.2           340           364   572    10
##  7 OKC            30.6          30.6           280           262   346     3
##  8 RIC            23.6          20.1           483           463  2454     8
##  9 TUL            34.9          33.7           251           262   315     2
## 10 TYS            28.5          24.1           291           281   631     5

Missing Values

Most summaries will be NA if any of the data values being summarized are NA.

Typically, summary functions can be called with na.rm = TRUE to produce the summary for all non-missing values.

summarize(flights,
          ave_dep_delay = mean(dep_delay, na.rm = TRUE),
          ave_arr_delay = mean(arr_delay, na.rm = TRUE),
          max_dep_delay = max(dep_delay, na.rm = TRUE),
          max_arr_delay = max(arr_delay, na.rm = TRUE),
          n = n())
## # A tibble: 1 × 5
##   ave_dep_delay ave_arr_delay max_dep_delay max_arr_delay      n
##           <dbl>         <dbl>         <dbl>         <dbl>  <int>
## 1          12.6          6.90          1301          1272 336776

You can also use filter to remove rows with NA values in the variables to be summarized:

filter(flights, ! is.na(dep_delay), ! is.na(arr_delay)) %>%
    summarize(ave_dep_delay = mean(dep_delay),
              ave_arr_delay = mean(arr_delay),
              max_dep_delay = max(dep_delay),
              max_arr_delay = max(arr_delay),
              n = n())
## # A tibble: 1 × 5
##   ave_dep_delay ave_arr_delay max_dep_delay max_arr_delay      n
##           <dbl>         <dbl>         <dbl>         <dbl>  <int>
## 1          12.6          6.90          1301          1272 327346

The two approaches do produce different counts.

Counts

The helper function n() provides a count of the number of rows in the table being summarized.

Including counts with grouped summaries is usually a good idea.

Summaries based on small counts are likely to be much less reliable than ones based on larger counts.

This is reflected in higher variability among averages for destinations with fewer flights:

ggplot(fl_dest) +
    geom_point(aes(x = n,
                   y = ave_arr_delay),
               na.rm = TRUE) +
    thm

A plotly version allows the individual destinations to be identified easily:

library(plotly)
p <- ggplot(fl_dest) +
    geom_point(aes(x = n,
                   y = ave_arr_delay,
                   text = dest),
               na.rm = FALSE) +
    thm
ggplotly(p, tooltip = "text")

It would be more user-friendly to show the airport name than the three letter FAA code.

Useful Summaries

Some useful summary functions:

  • Location: mean, median

  • Spread: sd, IQR, mad

  • Rank: min, quantile, max

  • Position: first, nth, last

  • Counts: n.

Airline Arrival Delays

Different Delay Measures

Sometimes it is useful to look at subsets or truncated versions of a variable.

Airlines like to arrange schedules so they are often early.

ggplot(flights, aes(x = arr_delay,
                    y = after_stat(density))) +
    geom_histogram(binwidth = 1,
                   na.rm = TRUE,
                   fill = "deepskyblue3") +
    xlim(c(-100, 250)) +
    thm

This reduces the average arrival delay but may not help travelers much.

Two options:

  • Consider only the positive delays for the average.

  • Treat negative delays as zero (zero-truncation).

The mean of the positive delays can be computed using subsetting:

mean(arr_delay[arr_delay > 0])

The mean of the zero-truncated delays can be computed using the pmax function:

mean(pmax(arr_delay, 0))

The summaries are

fl_arr <- filter(flights, ! is.na(arr_delay))
fl_avs <-
    group_by(fl_arr, dest) %>%
    summarize(ave = mean(arr_delay),
              ave_pos = mean(arr_delay[arr_delay > 0]),
              ave_pos_zero = mean(pmax(arr_delay, 0)),
              n = n()) %>%
    ungroup()
head(fl_avs)
## # A tibble: 6 × 5
##   dest    ave ave_pos ave_pos_zero     n
##   <chr> <dbl>   <dbl>        <dbl> <int>
## 1 ABQ    4.38    41.9        17.7    254
## 2 ACK    4.85    28.6        11.3    264
## 3 ALB   14.4     52.1        22.9    418
## 4 ANC   -2.5     12.4         7.75     8
## 5 ATL   11.3     37.8        17.8  16837
## 6 AUS    6.02    40.6        16.6   2411

One approach to visualizing the results with color coding the three different summaries:

  • View the fl_avs table as a wide format with avs, ave_pos, and ave_pos_zero as three different measurement types.

  • Use pivot_longer() to convert to a long format with a variable which to hold the summary type and a variable delay to hold the summary value:

library(tidyr)
fl <- pivot_longer(fl_avs, 2 : 4,
                   names_to = "which",
                   values_to = "delay")
arrange(fl, dest)
## # A tibble: 312 × 4
##    dest      n which        delay
##    <chr> <int> <chr>        <dbl>
##  1 ABQ     254 ave           4.38
##  2 ABQ     254 ave_pos      41.9 
##  3 ABQ     254 ave_pos_zero 17.7 
##  4 ACK     264 ave           4.85
##  5 ACK     264 ave_pos      28.6 
##  6 ACK     264 ave_pos_zero 11.3 
##  7 ALB     418 ave          14.4 
##  8 ALB     418 ave_pos      52.1 
##  9 ALB     418 ave_pos_zero 22.9 
## 10 ANC       8 ave          -2.5 
## # ℹ 302 more rows

A plot is then easy to create:

ggplot(fl, aes(x = n,
               y = delay,
               color = which)) +
    geom_point() +
    thm
## Warning: Removed 1 rows containing missing values (`geom_point()`).

An alternative:

  • Use one layer per summary type.

  • Specify the summary type for each layer as a color aesthetic value.

ggplot(fl_avs, aes(x = n)) +
    geom_point(aes(y = ave,
                   color = "ave")) +
    geom_point(aes(y = ave_pos,
                   color = "ave_pos")) +
    geom_point(aes(y = ave_pos_zero,
                   color = "ave_pos_zero")) +
    thm
## Warning: Removed 1 rows containing missing values (`geom_point()`).

The one missing value case:

incomplete_cases(fl_avs)
## # A tibble: 1 × 5
##   dest    ave ave_pos ave_pos_zero     n
##   <chr> <dbl>   <dbl>        <dbl> <int>
## 1 LEX     -22     NaN            0     1

Delay Proportions

Proportions satisfying a condition can be computed as a mean of the logical expression for the condition:

The proportion of all flights that were delayed:

mean(fl_arr$arr_delay > 0)
## [1] 0.4063101

Grouped by destination:

fl_props <-
    group_by(fl_arr, dest) %>%
    summarize(
        pd0 = mean(arr_delay > 0),   ## proportion delayed
        pd10 = mean(arr_delay > 10), ## more than 10 minutes
        pd20 = mean(arr_delay > 20), ## more than 20 minutes
        n = n()) %>%
    ungroup()
fl_props
## # A tibble: 104 × 5
##    dest    pd0  pd10  pd20     n
##    <chr> <dbl> <dbl> <dbl> <int>
##  1 ABQ   0.421 0.339 0.268   254
##  2 ACK   0.394 0.246 0.167   264
##  3 ALB   0.440 0.333 0.273   418
##  4 ANC   0.625 0.125 0.125     8
##  5 ATL   0.472 0.309 0.211 16837
##  6 AUS   0.408 0.284 0.216  2411
##  7 AVL   0.456 0.257 0.207   261
##  8 BDL   0.350 0.262 0.223   412
##  9 BGR   0.383 0.299 0.246   358
## 10 BHM   0.450 0.361 0.305   269
## # ℹ 94 more rows

These can be visualized similarly:

pivot_longer(fl_props, 2 : 4,
             names_to = "which",
             values_to = "prop") %>%
    ggplot(aes(x = n,
               y = prop,
               color = which)) +
    geom_point() +
    thm

Grouped Mutate and Filter

mutate(), filter(), arrange(), slice_max(), and slice_min() can be used on a group level.

Standardizing Within Groups

mutate within groups can be used for standardizing within a group.

For example, to compute the proportion of a continent’s population living in each country for each year:

gm <- group_by(gapminder, continent, year) %>%
    mutate(pop = as.numeric(pop),
           ppop = pop / sum(pop)) %>%
    ungroup()

For Oceania (in this data set) and years 1952 and 2007:

gm_oc <- filter(gm, continent == "Oceania", year %in% c(1952, 2007)) %>%
    arrange(year)
gm_oc
## # A tibble: 4 × 7
##   country     continent  year lifeExp      pop gdpPercap  ppop
##   <fct>       <fct>     <int>   <dbl>    <dbl>     <dbl> <dbl>
## 1 Australia   Oceania    1952    69.1  8691212    10040. 0.813
## 2 New Zealand Oceania    1952    69.4  1994794    10557. 0.187
## 3 Australia   Oceania    2007    81.2 20434176    34435. 0.832
## 4 New Zealand Oceania    2007    80.2  4115771    25185. 0.168

A quick check that that within year and continent the proportions add up to one:

group_by(gm_oc, continent, year) %>% summarize(sum_ppop = sum(ppop))
## # A tibble: 2 × 3
## # Groups:   continent [1]
##   continent  year sum_ppop
##   <fct>     <int>    <dbl>
## 1 Oceania    1952        1
## 2 Oceania    2007        1

Focus on the 2 countries in each continent with the highest proportions of the continent’s population:

gmt2 <- group_by(gm, continent, year) %>%
    slice_max(ppop, n = 2) %>%
    ungroup() %>%
    filter(continent != "Oceania")  ## drop as there are only two
ggplot(gmt2, aes(x = year, y = ppop, group = country, color = country)) +
    geom_line() + geom_point() +
    facet_wrap(~ continent, scales = "free_y") +
    thm

Another approach:

  • Create separate plots for each continent.

  • Adjust the themes a little.

  • Use facet_wrap to get the facet label.

  • Arrange the plots, e.g. using patchwork operators.

A useful feature is that the %+% operator can be used to change the base data frame for a plot:

pAf <- ggplot(filter(gmt2, continent == "Africa"),
              aes(x = year, y = ppop, col = country)) +
    geom_line() + geom_point() +
    facet_wrap(~ continent) +
    thm +
    theme(legend.position = "top", legend.title = element_blank())
thm_nx <- theme(axis.title.x = element_blank())
thm_ny <- theme(axis.title.y = element_blank())
pAm <- pAf %+% filter(gmt2, continent == "Americas")
pAs <- pAf %+% filter(gmt2, continent == "Asia")
pEu <- pAf %+% filter(gmt2, continent == "Europe") +
    guides(color = guide_legend(nrow = 2))
library(patchwork)
((pAf + thm_nx) | (pAm + thm_nx + thm_ny)) / (pAs | (pEu + thm_ny))

Changes Over Time

Changes over time within a country can also be examined with a grouped mutate and the lag operator.

For variables collected over time the lag function can be used to compute the previous value of a variable:

gm_us <- filter(gapminder,
                country == "United States") %>%
    transmute(year,
              lifeExp,
              prev_lifeExp = lag(lifeExp))
gm_us
## # A tibble: 12 × 3
##     year lifeExp prev_lifeExp
##    <int>   <dbl>        <dbl>
##  1  1952    68.4         NA  
##  2  1957    69.5         68.4
##  3  1962    70.2         69.5
##  4  1967    70.8         70.2
##  5  1972    71.3         70.8
##  6  1977    73.4         71.3
##  7  1982    74.6         73.4
##  8  1987    75.0         74.6
##  9  1992    76.1         75.0
## 10  1997    76.8         76.1
## 11  2002    77.3         76.8
## 12  2007    78.2         77.3

This can then be used to compute the change from one period to the next:

mutate(gm_us,
       le_change = lifeExp - prev_lifeExp)
## # A tibble: 12 × 4
##     year lifeExp prev_lifeExp le_change
##    <int>   <dbl>        <dbl>     <dbl>
##  1  1952    68.4         NA      NA    
##  2  1957    69.5         68.4     1.05 
##  3  1962    70.2         69.5     0.720
##  4  1967    70.8         70.2     0.550
##  5  1972    71.3         70.8     0.580
##  6  1977    73.4         71.3     2.04 
##  7  1982    74.6         73.4     1.27 
##  8  1987    75.0         74.6     0.370
##  9  1992    76.1         75.0     1.07 
## 10  1997    76.8         76.1     0.720
## 11  2002    77.3         76.8     0.5  
## 12  2007    78.2         77.3     0.932

Use this idea to compute changes in lifeExp and gdpPercap for all countries in a grouped mutate:

gm <- group_by(gapminder, country) %>%
    mutate(
        le_change = lifeExp - lag(lifeExp),
        gdp_change = gdpPercap - lag(gdpPercap),
        gdp_pct_change =
            100 * gdp_change / lag(gdpPercap)) %>%
    ungroup()

slice_max() can then be applied to data grouped by country to extract the rows with the highest gdp_pct_change for each country:

group_by(gm, country) %>%
    slice_max(gdp_pct_change, n = 1) %>%
    ungroup() %>%
    select(1 : 3, contains("gdp"))
## # A tibble: 142 × 6
##    country     continent  year gdpPercap gdp_change gdp_pct_change
##    <fct>       <fct>     <int>     <dbl>      <dbl>          <dbl>
##  1 Afghanistan Asia       2007      975.       248.           34.1
##  2 Albania     Europe     2002     4604.      1411.           44.2
##  3 Algeria     Africa     1972     4183.       936.           28.8
##  4 Angola      Africa     2007     4797.      2024.           73.0
##  5 Argentina   Americas   2007    12779.      3982.           45.3
##  6 Australia   Oceania    1967    14526.      2309.           18.9
##  7 Austria     Europe     1957     8843.      2706.           44.1
##  8 Bahrain     Asia       2007    29796.      6392.           27.3
##  9 Bangladesh  Asia       2007     1391.       255.           22.4
## 10 Belgium     Europe     1972    16672.      3523.           26.8
## # ℹ 132 more rows

Grouping by continent allows the worst drop in life expectancy for each continent to be found:

group_by(gm, continent) %>%
    slice_min(le_change, n = 1) %>%
    ungroup() %>%
    select(-contains("gdp"))
## # A tibble: 5 × 6
##   country     continent  year lifeExp      pop le_change
##   <fct>       <fct>     <int>   <dbl>    <int>     <dbl>
## 1 Rwanda      Africa     1992    23.6  7290203   -20.4  
## 2 El Salvador Americas   1977    56.7  4282586    -1.51 
## 3 Cambodia    Asia       1977    31.2  6978607    -9.10 
## 4 Montenegro  Europe     2002    74.0   720230    -1.46 
## 5 Australia   Oceania    1967    71.1 11872264     0.170

Without grouping, the row with best percent GDP change overall can be computed:

slice_max(gm, gdp_pct_change, n = 1) %>%
    select(country, continent, year, gdp_pct_change)
## # A tibble: 1 × 4
##   country continent  year gdp_pct_change
##   <fct>   <fct>     <int>          <dbl>
## 1 Libya   Africa     1967           178.

The computation of the best GDP growth years by country can be combined into a single pipe:

group_by(gapminder, country) %>%
    mutate(gdp_change = gdpPercap - lag(gdpPercap),
           gdp_pct_change = 100 * gdp_change / lag(gdpPercap)) %>%
    ungroup() %>%
    group_by(country) %>%
    slice_max(gdp_pct_change, n = 1) %>%
    ungroup() %>%
    select(1 : 3, contains("gdp"))

Since the groupings of the two steps are the same and grouped mutate preserves the group structure, the middle ungroup/group_by can be dropped:

group_by(gapminder, country) %>%
    mutate(gdp_change = gdpPercap - lag(gdpPercap),
           gdp_pct_change = 100 * gdp_change / lag(gdpPercap)) %>%
    slice_max(gdp_pct_change, n = 1) %>%
    ungroup() %>%
    select(1 : 3, contains("gdp"))

You need to be very careful making an optimization like this!

Joins

This interactive plot uses the tooltip to show the airport code for the destination:

library(plotly)
p <- ggplot(fl_dest) +
    geom_point(aes(x = n,
                   y = ave_arr_delay,
                   text = dest)) +
    thm
## Warning in geom_point(aes(x = n, y = ave_arr_delay, text = dest)): Ignoring
## unknown aesthetics: text
ggplotly(p, tooltip = "text")

A nicer approach would be to show the airport name.

The airports table provides this information:

head(airports, 2)
## # A tibble: 2 × 8
##   faa   name                            lat   lon   alt    tz dst   tzone       
##   <chr> <chr>                         <dbl> <dbl> <dbl> <dbl> <chr> <chr>       
## 1 04G   Lansdowne Airport              41.1 -80.6  1044    -5 A     America/New…
## 2 06A   Moton Field Municipal Airport  32.5 -85.7   264    -6 A     America/Chi…

We need to get this information into the fl_dest data frame.

As another example, we might want to explore whether aircraft age is related to delays.

The planes table provides the year of manufacture:

head(planes, 2)
## # A tibble: 2 × 9
##   tailnum  year type               manufacturer model engines seats speed engine
##   <chr>   <int> <chr>              <chr>        <chr>   <int> <int> <int> <chr> 
## 1 N10156   2004 Fixed wing multi … EMBRAER      EMB-…       2    55    NA Turbo…
## 2 N102UW   1998 Fixed wing multi … AIRBUS INDU… A320…       2   182    NA Turbo…

Again, we need to merge this information with the data in the flights table.

One way to do this is with a join operation.

Joins are a common operation in data bases.

dplyr distinguishes between two kinds of joins:

These animated explanations may be helpful.

Mutating Joins

Mutating joins combine variables and rows from two tables by matching rows on keys.

  • A primary key is a variable, or combination of variables, in a table that uniquely identify the rows of the table.

  • A foreign key is a variable, or combination of variables, that uniquely identify a row in some table (usually, but not necessarily, a different table).

The simplest join operation is an inner join: Only rows for which the key appears in both tables are retained.

Outer joins retain some other rows with NA values for new variables:

Inner and left joins are the most common.

The key of one of the tables should usually be a primary key that uniquely identifies the table’s rows.

For a left join, the right table key should usually use a primary key.

It is usually a good idea to make sure a primary key candidate really does uniquely identify rows.

For the planes table the tailnum variable is a primary key;

count(planes, tailnum) %>% filter(n > 1)
## # A tibble: 0 × 2
## # ℹ 2 variables: tailnum <chr>, n <int>

For the flights data a primary key needs to use multiple variables.

A reasonable guess is to use the date, carrier and flight number, but this isn’t quite enough:

count(flights, year, month, day, flight, carrier) %>%
    filter(n > 1) %>%
    head(2)
## # A tibble: 2 × 6
##    year month   day flight carrier     n
##   <int> <int> <int>  <int> <chr>   <int>
## 1  2013     6     8   2269 WN          2
## 2  2013     6    15   2269 WN          2

Adding the origin airport code does produce a primary key:

count(flights, year, month, day, flight, carrier, origin) %>%
    filter(n > 1)
## # A tibble: 0 × 7
## # ℹ 7 variables: year <int>, month <int>, day <int>, flight <int>,
## #   carrier <chr>, origin <chr>, n <int>

To relate delays to aircraft age we can start with summaries by tailnum and year (year would be needed if we had data for more than 2013):

fl <- filter(flights, ! is.na(dep_time), ! is.na(arr_time)) %>%
    group_by(year, tailnum) %>%
    summarize(ave_dep_delay = mean(dep_delay, na.rm = TRUE),
              ave_arr_delay = mean(arr_delay, na.rm = TRUE),
              n = n()) %>%
    ungroup()
head(fl)
## # A tibble: 6 × 5
##    year tailnum ave_dep_delay ave_arr_delay     n
##   <int> <chr>           <dbl>         <dbl> <int>
## 1  2013 D942DN          31.5          31.5      4
## 2  2013 N0EGMQ           8.49          9.98   354
## 3  2013 N10156          17.8          12.7    146
## 4  2013 N102UW           8             2.94    48
## 5  2013 N103US          -3.20         -6.93    46
## 6  2013 N104UW          10.1           1.80    46

A reduced planes table with year renamed to avoid a conflict:

pl <- select(planes, tailnum, plane_year = year)
pl
## # A tibble: 3,322 × 2
##    tailnum plane_year
##    <chr>        <int>
##  1 N10156        2004
##  2 N102UW        1998
##  3 N103US        1999
##  4 N104UW        1999
##  5 N10575        2002
##  6 N105UW        1999
##  7 N107US        1999
##  8 N108UW        1999
##  9 N109UW        1999
## 10 N110UW        1999
## # ℹ 3,312 more rows

Now

  • use a left join with tailnum as the key to bring the plane_year value into the summary table

  • and then compute the plane age:

fl_age <- left_join(fl, pl, "tailnum") %>%
    mutate(plane_age = year - plane_year)
fl_age
## # A tibble: 4,037 × 7
##     year tailnum ave_dep_delay ave_arr_delay     n plane_year plane_age
##    <int> <chr>           <dbl>         <dbl> <int>      <int>     <int>
##  1  2013 D942DN         31.5          31.5       4         NA        NA
##  2  2013 N0EGMQ          8.49          9.98    354         NA        NA
##  3  2013 N10156         17.8          12.7     146       2004         9
##  4  2013 N102UW          8             2.94     48       1998        15
##  5  2013 N103US         -3.20         -6.93     46       1999        14
##  6  2013 N104UW         10.1           1.80     46       1999        14
##  7  2013 N10575         22.3          20.7     271       2002        11
##  8  2013 N105UW          2.58         -0.267    45       1999        14
##  9  2013 N107US         -0.463        -5.73     41       1999        14
## 10  2013 N108UW          4.22         -1.25     60       1999        14
## # ℹ 4,027 more rows

An initial plot:

ggplot(fl_age,
       aes(x = plane_age,
           y = ave_dep_delay)) +
    geom_point() +
    thm

Jittering and adjusting the point size may help:

ggplot(fl_age,
       aes(x = plane_age,
           y = ave_dep_delay)) +
    geom_point(position = "jitter",
               size = 0.1) +
    thm

Adding a smooth and adjusting the ranges may also help:

ggplot(fl_age,
       aes(x = plane_age,
           y = ave_dep_delay)) +
    geom_point(position = "jitter",
               size = 0.1) +
    geom_smooth() +
    xlim(0, 30) +
    ylim(c(-20, 80)) +
    thm

Another option is to compute means or medians within years:

flm <- group_by(fl_age, plane_age) %>%
    summarize(ave_dep_delay =
                  mean(ave_dep_delay,
                       na.rm = TRUE)) %>%
    ungroup()
ggplot(flm,
       aes(x = plane_age,
           y = ave_dep_delay)) +
    geom_point(na.rm = TRUE) +
    thm

Both the smooth and the means show a modest increase over the first 10 years.

A left join can also be used to add the airport name from the airport table to the fl_dest table.

The primary key in the airport table is the three-letter airport code in the faa variable:

head(airports, 2)
## # A tibble: 2 × 8
##   faa   name                            lat   lon   alt    tz dst   tzone       
##   <chr> <chr>                         <dbl> <dbl> <dbl> <dbl> <chr> <chr>       
## 1 04G   Lansdowne Airport              41.1 -80.6  1044    -5 A     America/New…
## 2 06A   Moton Field Municipal Airport  32.5 -85.7   264    -6 A     America/Chi…

The foreign key in the fl_dest table to match is the dest variable:

head(fl_dest, 2)
## # A tibble: 2 × 6
##   dest  ave_dep_delay ave_arr_delay max_dep_delay max_arr_delay     n
##   <chr>         <dbl>         <dbl>         <dbl>         <dbl> <int>
## 1 ABQ           13.7           4.38           142           153   254
## 2 ACK            6.46          4.85           219           221   265

The left_join function allows this link to be made by specifying the key as c("dest" = "faa"):

fl_dest <- left_join(fl_dest,
                     select(airports, faa, dest_name = name),
                     c("dest" = "faa"))
select(fl_dest, dest_name, everything()) %>% head(2)
## # A tibble: 2 × 7
##   dest_name  dest  ave_dep_delay ave_arr_delay max_dep_delay max_arr_delay     n
##   <chr>      <chr>         <dbl>         <dbl>         <dbl>         <dbl> <int>
## 1 Albuquerq… ABQ           13.7           4.38           142           153   254
## 2 Nantucket… ACK            6.46          4.85           219           221   265

Now a tooltip can provide a more accessible label:

p <- ggplot(fl_dest) +
    geom_point(aes(x = n,
                   y = ave_arr_delay,
                   text = paste(dest,
                                dest_name,
                                sep = ": "))) +
    thm
ggplotly(p, tooltip = "text")

Filtering Joins

Filtering joins only affect rows; they do not add variables.

  • semi_join(x, y) returns all rows in x with a match in y:

  • anti_join(x, y) returns all rows in x without a match in y:

semi_join is useful for matching filtered results back to a table.

The plot of delay against age showed some surprisingly old aircraft:

top_age <- slice_max(fl_age, plane_age, n = 2)
top_age
## # A tibble: 3 × 7
##    year tailnum ave_dep_delay ave_arr_delay     n plane_year plane_age
##   <int> <chr>           <dbl>         <dbl> <int>      <int>     <int>
## 1  2013 N381AA           5.95          3.5     22       1956        57
## 2  2013 N201AA          17.4          14.1     51       1959        54
## 3  2013 N567AA           3.72         -4.30    61       1959        54

We can identify these aircraft with a semi-join:

semi_join(planes, top_age, "tailnum")
## # A tibble: 3 × 9
##   tailnum  year type               manufacturer model engines seats speed engine
##   <chr>   <int> <chr>              <chr>        <chr>   <int> <int> <int> <chr> 
## 1 N201AA   1959 Fixed wing single… CESSNA       150         1     2    90 Recip…
## 2 N381AA   1956 Fixed wing multi … DOUGLAS      DC-7…       4   102   232 Recip…
## 3 N567AA   1959 Fixed wing single… DEHAVILLAND  OTTE…       1    16    95 Recip…

The corresponding flights can also be picked out with a semi-join:

top_age_flights <- semi_join(flights, top_age, "tailnum")
head(top_age_flights, 2)
## # A tibble: 2 × 19
##    year month   day dep_time sched_dep_time dep_delay arr_time sched_arr_time
##   <int> <int> <int>    <int>          <int>     <dbl>    <int>          <int>
## 1  2013     1     3      909            700       129     1103            850
## 2  2013     1     3       NA            920        NA       NA           1245
## # ℹ 11 more variables: arr_delay <dbl>, carrier <chr>, flight <int>,
## #   tailnum <chr>, origin <chr>, dest <chr>, air_time <dbl>, distance <dbl>,
## #   hour <dbl>, minute <dbl>, time_hour <dttm>

Looking at the carriers or destinations might help:

select(top_age_flights, carrier, dest, everything()) %>% head(4)
## # A tibble: 4 × 19
##   carrier dest   year month   day dep_time sched_dep_time dep_delay arr_time
##   <chr>   <chr> <int> <int> <int>    <int>          <int>     <dbl>    <int>
## 1 AA      ORD    2013     1     3      909            700       129     1103
## 2 AA      DFW    2013     1     3       NA            920        NA       NA
## 3 AA      DFW    2013     1     9     1423           1430        -7     1727
## 4 AA      DFW    2013     1    10     1238           1240        -2     1525
## # ℹ 10 more variables: sched_arr_time <int>, arr_delay <dbl>, flight <int>,
## #   tailnum <chr>, origin <chr>, air_time <dbl>, distance <dbl>, hour <dbl>,
## #   minute <dbl>, time_hour <dttm>

The carrier summary:

count(top_age_flights, carrier)
## # A tibble: 1 × 2
##   carrier     n
##   <chr>   <int>
## 1 AA        139

A check of the documentation for planes shows that American and one other airline report fleet numbers rather than tail numbers.

Anti-joins are useful for detecting join problems.

For example, many flights have tail numbers that do not match entries in planes:

fl <- anti_join(flights, planes, "tailnum")
nrow(fl)
## [1] 52606
count(fl, tailnum)
## # A tibble: 722 × 2
##    tailnum     n
##    <chr>   <int>
##  1 D942DN      4
##  2 N0EGMQ    371
##  3 N14628      1
##  4 N149AT     22
##  5 N16632     11
##  6 N17627      2
##  7 N1EAMQ    210
##  8 N200AA     34
##  9 N24633      8
## 10 N261AV      6
## # ℹ 712 more rows

Join Issues

  • Make sure a primary key you want to use has no missing values.

  • Make sure a primary key you want to use really uniquely identifies a row.

  • Make sure every value of your foreign key you want to use to link to another table matches a row in the linked table (anti_join is a good way to do this).

  • Check that your foreign key has no missing values.

Data Base Interface

Current data on airline flights in the US is available from the Bureau of Transportation Statistics at

http://www.transtats.bts.gov/DL_SelectFields.asp?Table_ID=236

The data are available in one compressed CSV file per month.

Loaded into R, one month of data requires about 300 Mb of memory.

Data for 1987–2008 was compiled for a DataExpo competition of the American Statistical Association held in 2009.

The data are available on the Linux systems in the directory /group/statsoft/data/DataExpo09 as compressed CSV files and as a SQLite data base.

Loading the data into R would require about 70 Gb of memory, which is not practical.

Using the data base is a good alternative.

SQLite is a simple relational data base that works with data stored on a local disk.

More sophisticated data bases allow data to be stored on multiple machines or in the cloud

These data bases are usually accessed by connecting to a server, which usually involves an authentication process.

For SQLite the connection is set up by linking to a file.

The RSQLite package provides an interface to SQLite.

The DBI package provides a high level interface for using data bases.

Analogous packages exist for Python and Perl.

The first step is to connect to the data base.

library(dplyr)
library(DBI)
library(RSQLite)
db <- dbConnect(SQLite(), "/group/statsoft/data/DataExpo09/ontime.sqlite3")

There is only one table:

dbListTables(db)
## [1] "ontime"

tbl creates a tibble-like object for a data base table.

fl <- tbl(db, "ontime")

This object is not a regular tibble, but many tibble operations work on it.

To find the variables in the table:

tbl_vars(fl)
##  [1] "Year"              "Month"             "DayofMonth"
##  [4] "DayOfWeek"         "DepTime"           "CRSDepTime"
##  [7] "ArrTime"           "CRSArrTime"        "UniqueCarrier"
## [10] "FlightNum"         "TailNum"           "ActualElapsedTime"
## [13] "CRSElapsedTime"    "AirTime"           "ArrDelay"
## [16] "DepDelay"          "Origin"            "Dest"
## [19] "Distance"          "TaxiIn"            "TaxiOut"
## [22] "Cancelled"         "CancellationCode"  "Diverted"
## [25] "CarrierDelay"      "WeatherDelay"      "NASDelay"
## [28] "SecurityDelay"     "LateAircraftDelay"

Many dplyr operations are supported:

sm <- summarize(fl, last_year = max(year), n = n())

This computation is fast because it does not actually access the data base.

An operation that needs the data, such as printing the result, will access the data base and take time:

sm
## # Source:   lazy query [?? x 2]
## # Database: sqlite 3.37.2
## #   [/mnt/nfs/clasnetappvm/group/statsoft/data/DataExpo09/ontime.sqlite3]
##   last_year         n
##       <int>     <int>
## 1      2008 118914458

The sm object essentially contains a SQL query, which can be examined using sql_render from the dbplyr package:

dbplyr::sql_render(sm)
## <SQL> SELECT MAX(`year`) AS `last_year`, COUNT(*) AS `n`
## FROM `ontime`

Flights to CID in 2008:

fl_cid <- filter(fl, year == 2008, Dest == "CID")
dbplyr::sql_render(fl_cid)
## <SQL> SELECT *
## FROM `ontime`
## WHERE ((`year` = 2008.0) AND (`Dest` = 'CID'))

Number of flights to CID in 2008:

sm <- summarize(fl_cid, n = n())
dbplyr::sql_render(sm)
## <SQL> SELECT COUNT(*) AS `n`
## FROM `ontime`
## WHERE ((`year` = 2008.0) AND (`Dest` = 'CID'))
sm
## # Source:   lazy query [?? x 1]
## # Database: sqlite 3.37.2
## #   [/mnt/nfs/clasnetappvm/group/statsoft/data/DataExpo09/ontime.sqlite3]
##       n
##   <int>
## 1  2961

Breakdown by where the flights originated:

sm <- group_by(fl_cid, Origin) %>%
    summarize(n = n()) %>%
    ungroup()
dbplyr::sql_render(sm)
## <SQL> SELECT `Origin`, COUNT(*) AS `n`
## FROM `ontime`
## WHERE ((`year` = 2008.0) AND (`Dest` = 'CID'))
## GROUP BY `Origin`
sm
## # Source:   lazy query [?? x 2]
## # Database: sqlite 3.37.2
## #   [/mnt/nfs/clasnetappvm/group/statsoft/data/DataExpo09/ontime.sqlite3]
##   Origin     n
##   <chr>  <int>
## 1 ATL      119
## 2 BNA        1
## 3 CVG      292
## 4 DEN      230
## 5 DFW      519
## 6 DTW      345
## 7 MSP      241
## 8 ORD     1214

Many base R functions can be converted to SQL:

sm <- group_by(fl_cid, Origin) %>%
    summarize(ave_arr_del_pos = mean(pmax(ArrDelay, 0), na.rm = TRUE)) %>%
    ungroup()
dbplyr::sql_render(sm)
## <SQL> SELECT `Origin`, AVG(MAX(`ArrDelay`, 0.0)) AS `ave_arr_del_pos`
## FROM `ontime`
## WHERE ((`year` = 2008.0) AND (`Dest` = 'CID'))
## GROUP BY `Origin`
sm_cid <- sm
sm
## # Source:   lazy query [?? x 2]
## # Database: sqlite 3.37.2
## #   [/mnt/nfs/clasnetappvm/group/statsoft/data/DataExpo09/ontime.sqlite3]
##   Origin ave_arr_del_pos
##   <chr>            <dbl>
## 1 ATL               17.4
## 2 BNA               18
## 3 CVG               12.4
## 4 DEN               12.1
## 5 DFW               11.3
## 6 DTW               15.0
## 7 MSP               20.1
## 8 ORD               21.6

mutate() can also be used:

fl_cid_gain <- mutate(fl_cid, Gain = DepDelay - ArrDelay)
dbplyr::sql_render(fl_cid_gain)
## <SQL> SELECT `Year`, `Month`, `DayofMonth`, `DayOfWeek`, `DepTime`, `CRSDepTime`, `ArrTime`, `CRSArrTime`, `UniqueCarrier`, `FlightNum`, `TailNum`, `ActualElapsedTime`, `CRSElapsedTime`, `AirTime`, `ArrDelay`, `DepDelay`, `Origin`, `Dest`, `Distance`, `TaxiIn`, `TaxiOut`, `Cancelled`, `CancellationCode`, `Diverted`, `CarrierDelay`, `WeatherDelay`, `NASDelay`, `SecurityDelay`, `LateAircraftDelay`, `DepDelay` - `ArrDelay` AS `Gain`
## FROM `ontime`
## WHERE ((`year` = 2008.0) AND (`Dest` = 'CID'))
group_by(fl_cid_gain, Origin) %>%
    summarize(mean_gain = mean(Gain)) %>%
    ungroup()
## # Source:   lazy query [?? x 2]
## # Database: sqlite 3.37.2
## #   [/mnt/nfs/clasnetappvm/group/statsoft/data/DataExpo09/ontime.sqlite3]
##   Origin mean_gain
##   <chr>      <dbl>
## 1 ATL        8.24
## 2 BNA      -26
## 3 CVG       -1.60
## 4 DEN        0.278
## 5 DFW        4.12
## 6 DTW        3.04
## 7 MSP       -0.357
## 8 ORD       -1.36

Once a filtered result is small enough it can be brought over as a regular tibble using

tb <- collect(fl_cid)

There are some limitations to operations that can be done on tables in data bases.

Defining and using a function works for a tibble:

pos_mean <- function(x) mean(pmax(x, 0))
group_by(tb, Origin) %>%
    summarize(ave_arr_del_pos = pos_mean(ArrDelay)) %>%
    ungroup()

But this fails:

group_by(fl_cid, Origin) %>%
    summarize(ave_arr_del_pos = pos_mean(ArrDelay)) %>%
    ungroup()
## Error: no such function: pos_mean

Some operations may require some modifications.

Recomputing sm_cid:

fl <- tbl(db, "ontime")
fl_cid <- filter(fl, year == 2008, Dest == "CID")
sm_cid <- group_by(fl_cid, Origin) %>%
    summarize(ave_arr_del_pos = mean(pmax(ArrDelay, 0), na.rm = TRUE)) %>%
    ungroup()

Joining with a local table does not work:

ap <- select(nycflights13::airports, faa, name)
sm_cid <- left_join(sm_cid, ap, c(Origin = "faa"))
## Error in `auto_copy()`:
## ! `x` and `y` must share the same src.
## ℹ set `copy` = TRUE (may be slow).

But this does work:

sm_cid <- left_join(sm_cid, ap, c(Origin = "faa"), copy = TRUE)

This copies the ap table to the data base:

dbListTables(db)
## [1] "dbplyr_001"   "ontime"       "sqlite_stat1" "sqlite_stat4"

Finish by disconnecting from the data base:

dbDisconnect(db)

This is not necessary for SQLite but is for other data bases, so it is good practice.

Some notes:

The dbplyr web site provides more information on using data bases with dplyr.

Other Wrangling Tools

Some additional tools provided by tidyr:

We have already seen pivot_longer() and pivot_wider().

Separating Variables

A data set from the WHO on tuberculosis cases:

head(who, 2)
## # A tibble: 2 × 60
##   country   iso2  iso3   year new_sp_m014 new_sp_m1524 new_sp_m2534 new_sp_m3544
##   <chr>     <chr> <chr> <dbl>       <dbl>        <dbl>        <dbl>        <dbl>
## 1 Afghanis… AF    AFG    1980          NA           NA           NA           NA
## 2 Afghanis… AF    AFG    1981          NA           NA           NA           NA
## # ℹ 52 more variables: new_sp_m4554 <dbl>, new_sp_m5564 <dbl>,
## #   new_sp_m65 <dbl>, new_sp_f014 <dbl>, new_sp_f1524 <dbl>,
## #   new_sp_f2534 <dbl>, new_sp_f3544 <dbl>, new_sp_f4554 <dbl>,
## #   new_sp_f5564 <dbl>, new_sp_f65 <dbl>, new_sn_m014 <dbl>,
## #   new_sn_m1524 <dbl>, new_sn_m2534 <dbl>, new_sn_m3544 <dbl>,
## #   new_sn_m4554 <dbl>, new_sn_m5564 <dbl>, new_sn_m65 <dbl>,
## #   new_sn_f014 <dbl>, new_sn_f1524 <dbl>, new_sn_f2534 <dbl>, …

Some of the remaining variable names:

names(who) %>% tail(20)
##  [1] "new_ep_f1524" "new_ep_f2534" "new_ep_f3544" "new_ep_f4554" "new_ep_f5564"
##  [6] "new_ep_f65"   "newrel_m014"  "newrel_m1524" "newrel_m2534" "newrel_m3544"
## [11] "newrel_m4554" "newrel_m5564" "newrel_m65"   "newrel_f014"  "newrel_f1524"
## [16] "newrel_f2534" "newrel_f3544" "newrel_f4554" "newrel_f5564" "newrel_f65"

The variable new_sp_m2534, for example, represents the number of cases for

  • type sp (positive pulmonary smear);
  • sex m;
  • age between 25 and 34.

It would be better (tidier) to have separate variables for

  • count
  • type
  • sex
  • lower bound on age bracket
  • upper bound on age bracket

A pipeline to clean this up involves pivoting to long form and several separate and mutate steps.

The original data frame:

who
## # A tibble: 7,240 × 60
##    country  iso2  iso3   year new_sp_m014 new_sp_m1524 new_sp_m2534 new_sp_m3544
##    <chr>    <chr> <chr> <dbl>       <dbl>        <dbl>        <dbl>        <dbl>
##  1 Afghani… AF    AFG    1980          NA           NA           NA           NA
##  2 Afghani… AF    AFG    1981          NA           NA           NA           NA
##  3 Afghani… AF    AFG    1982          NA           NA           NA           NA
##  4 Afghani… AF    AFG    1983          NA           NA           NA           NA
##  5 Afghani… AF    AFG    1984          NA           NA           NA           NA
##  6 Afghani… AF    AFG    1985          NA           NA           NA           NA
##  7 Afghani… AF    AFG    1986          NA           NA           NA           NA
##  8 Afghani… AF    AFG    1987          NA           NA           NA           NA
##  9 Afghani… AF    AFG    1988          NA           NA           NA           NA
## 10 Afghani… AF    AFG    1989          NA           NA           NA           NA
## # ℹ 7,230 more rows
## # ℹ 52 more variables: new_sp_m4554 <dbl>, new_sp_m5564 <dbl>,
## #   new_sp_m65 <dbl>, new_sp_f014 <dbl>, new_sp_f1524 <dbl>,
## #   new_sp_f2534 <dbl>, new_sp_f3544 <dbl>, new_sp_f4554 <dbl>,
## #   new_sp_f5564 <dbl>, new_sp_f65 <dbl>, new_sn_m014 <dbl>,
## #   new_sn_m1524 <dbl>, new_sn_m2534 <dbl>, new_sn_m3544 <dbl>,
## #   new_sn_m4554 <dbl>, new_sn_m5564 <dbl>, new_sn_m65 <dbl>, …

First pivot to longer form:

who_clean <-
    pivot_longer(who,
                 new_sp_m014 : newrel_f65,
                 names_to = "key",
                 values_to = "count")
who_clean
## # A tibble: 405,440 × 6
##    country     iso2  iso3   year key          count
##    <chr>       <chr> <chr> <dbl> <chr>        <dbl>
##  1 Afghanistan AF    AFG    1980 new_sp_m014     NA
##  2 Afghanistan AF    AFG    1980 new_sp_m1524    NA
##  3 Afghanistan AF    AFG    1980 new_sp_m2534    NA
##  4 Afghanistan AF    AFG    1980 new_sp_m3544    NA
##  5 Afghanistan AF    AFG    1980 new_sp_m4554    NA
##  6 Afghanistan AF    AFG    1980 new_sp_m5564    NA
##  7 Afghanistan AF    AFG    1980 new_sp_m65      NA
##  8 Afghanistan AF    AFG    1980 new_sp_f014     NA
##  9 Afghanistan AF    AFG    1980 new_sp_f1524    NA
## 10 Afghanistan AF    AFG    1980 new_sp_f2534    NA
## # ℹ 405,430 more rows

Remove "new" or "new_" prefix:

who_clean <-
    pivot_longer(who,
                 new_sp_m014 : newrel_f65,
                 names_to = "key",
                 values_to = "count") %>%
    mutate(key = sub("new_?", "", key))
who_clean
## # A tibble: 405,440 × 6
##    country     iso2  iso3   year key      count
##    <chr>       <chr> <chr> <dbl> <chr>    <dbl>
##  1 Afghanistan AF    AFG    1980 sp_m014     NA
##  2 Afghanistan AF    AFG    1980 sp_m1524    NA
##  3 Afghanistan AF    AFG    1980 sp_m2534    NA
##  4 Afghanistan AF    AFG    1980 sp_m3544    NA
##  5 Afghanistan AF    AFG    1980 sp_m4554    NA
##  6 Afghanistan AF    AFG    1980 sp_m5564    NA
##  7 Afghanistan AF    AFG    1980 sp_m65      NA
##  8 Afghanistan AF    AFG    1980 sp_f014     NA
##  9 Afghanistan AF    AFG    1980 sp_f1524    NA
## 10 Afghanistan AF    AFG    1980 sp_f2534    NA
## # ℹ 405,430 more rows

Separate key into type and sexage:

who_clean <-
    pivot_longer(who,
                 new_sp_m014 : newrel_f65,
                 names_to = "key",
                 values_to = "count") %>%
    mutate(key = sub("new_?", "", key)) %>%
    separate(key, c("type", "sexage"))
who_clean
## # A tibble: 405,440 × 7
##    country     iso2  iso3   year type  sexage count
##    <chr>       <chr> <chr> <dbl> <chr> <chr>  <dbl>
##  1 Afghanistan AF    AFG    1980 sp    m014      NA
##  2 Afghanistan AF    AFG    1980 sp    m1524     NA
##  3 Afghanistan AF    AFG    1980 sp    m2534     NA
##  4 Afghanistan AF    AFG    1980 sp    m3544     NA
##  5 Afghanistan AF    AFG    1980 sp    m4554     NA
##  6 Afghanistan AF    AFG    1980 sp    m5564     NA
##  7 Afghanistan AF    AFG    1980 sp    m65       NA
##  8 Afghanistan AF    AFG    1980 sp    f014      NA
##  9 Afghanistan AF    AFG    1980 sp    f1524     NA
## 10 Afghanistan AF    AFG    1980 sp    f2534     NA
## # ℹ 405,430 more rows

Separate sexage into sex and age:

who_clean <-
    pivot_longer(who,
                 new_sp_m014 : newrel_f65,
                 names_to = "key",
                 values_to = "count") %>%
    mutate(key = sub("new_?", "", key)) %>%
    separate(key, c("type", "sexage")) %>%
    separate(sexage, c("sex", "age"),
             sep = 1)
who_clean
## # A tibble: 405,440 × 8
##    country     iso2  iso3   year type  sex   age   count
##    <chr>       <chr> <chr> <dbl> <chr> <chr> <chr> <dbl>
##  1 Afghanistan AF    AFG    1980 sp    m     014      NA
##  2 Afghanistan AF    AFG    1980 sp    m     1524     NA
##  3 Afghanistan AF    AFG    1980 sp    m     2534     NA
##  4 Afghanistan AF    AFG    1980 sp    m     3544     NA
##  5 Afghanistan AF    AFG    1980 sp    m     4554     NA
##  6 Afghanistan AF    AFG    1980 sp    m     5564     NA
##  7 Afghanistan AF    AFG    1980 sp    m     65       NA
##  8 Afghanistan AF    AFG    1980 sp    f     014      NA
##  9 Afghanistan AF    AFG    1980 sp    f     1524     NA
## 10 Afghanistan AF    AFG    1980 sp    f     2534     NA
## # ℹ 405,430 more rows

Fix up age categories:

who_clean <-
    pivot_longer(who,
                 new_sp_m014 : newrel_f65,
                 names_to = "key",
                 values_to = "count") %>%
    mutate(key = sub("new_?", "", key)) %>%
    separate(key, c("type", "sexage")) %>%
    separate(sexage, c("sex", "age"),
             sep = 1) %>%
    mutate(age = sub("014", "0014", age)) %>%
    mutate(age = sub("65", "65Inf", age))
who_clean
## # A tibble: 405,440 × 8
##    country     iso2  iso3   year type  sex   age   count
##    <chr>       <chr> <chr> <dbl> <chr> <chr> <chr> <dbl>
##  1 Afghanistan AF    AFG    1980 sp    m     0014     NA
##  2 Afghanistan AF    AFG    1980 sp    m     1524     NA
##  3 Afghanistan AF    AFG    1980 sp    m     2534     NA
##  4 Afghanistan AF    AFG    1980 sp    m     3544     NA
##  5 Afghanistan AF    AFG    1980 sp    m     4554     NA
##  6 Afghanistan AF    AFG    1980 sp    m     5564     NA
##  7 Afghanistan AF    AFG    1980 sp    m     65Inf    NA
##  8 Afghanistan AF    AFG    1980 sp    f     0014     NA
##  9 Afghanistan AF    AFG    1980 sp    f     1524     NA
## 10 Afghanistan AF    AFG    1980 sp    f     2534     NA
## # ℹ 405,430 more rows

Finally, split age into age_lo and age-hi:

who_clean <-
    pivot_longer(who,
                 new_sp_m014 : newrel_f65,
                 names_to = "key",
                 values_to = "count") %>%
    mutate(key = sub("new_?", "", key)) %>%
    separate(key, c("type", "sexage")) %>%
    separate(sexage, c("sex", "age"),
             sep = 1) %>%
    mutate(age = sub("014", "0014", age)) %>%
    mutate(age = sub("65", "65Inf", age)) %>%
    separate(age, c("age_lo", "age_hi"),
             sep = 2,
             convert = TRUE)
who_clean
## # A tibble: 405,440 × 9
##    country     iso2  iso3   year type  sex   age_lo age_hi count
##    <chr>       <chr> <chr> <dbl> <chr> <chr>  <int>  <dbl> <dbl>
##  1 Afghanistan AF    AFG    1980 sp    m          0     14    NA
##  2 Afghanistan AF    AFG    1980 sp    m         15     24    NA
##  3 Afghanistan AF    AFG    1980 sp    m         25     34    NA
##  4 Afghanistan AF    AFG    1980 sp    m         35     44    NA
##  5 Afghanistan AF    AFG    1980 sp    m         45     54    NA
##  6 Afghanistan AF    AFG    1980 sp    m         55     64    NA
##  7 Afghanistan AF    AFG    1980 sp    m         65    Inf    NA
##  8 Afghanistan AF    AFG    1980 sp    f          0     14    NA
##  9 Afghanistan AF    AFG    1980 sp    f         15     24    NA
## 10 Afghanistan AF    AFG    1980 sp    f         25     34    NA
## # ℹ 405,430 more rows

A plot of total numbers of cases for a few countries over time:

filter(who_clean,
       iso3 %in% c("CAN", "DEU", "GBR", "USA")) %>%
    group_by(year, iso3) %>%
    summarize(count = sum(count, na.rm = TRUE)) %>%
    ungroup() %>%
    ggplot() +
    geom_line(aes(x = year,
                  y = count,
                  color = iso3)) +
    thm

Filling In Omitted Values

The Refugee Processing Center provides information from the United States Refugee Admissions Program (USRAP), including arrivals by state and nationality.

Three files are available locally:

These are Excel spread sheets.

Reading in the third file, with data for October 2017 through February 2018:

ref_url <-
    "http://homepage.stat.uiowa.edu/~luke/data/Arrivals-2018-03-05.xls"
ref_file <- "Arrivals-2018-03-05.xls"
if (! file.exists(ref_file))
    download.file(ref_url, ref_file)

library(readxl)
ref <- read_excel(ref_file, skip = 16) ## skip header
ref <- head(ref, -2) ## drop last two rows
names(ref)[1] <- "Destination"

The Destination column needs filling in (this is common in spread sheet data):

ref
## # A tibble: 500 × 18
##    Destination Nationality   Oct   Nov   Dec   Jan   Feb   Mar   Apr   May   Jun
##    <chr>       <chr>       <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
##  1 Alabama     <NA>            0     0     3     1     4     0     0     0     0
##  2 <NA>        Dem. Rep. …     0     0     3     0     4     0     0     0     0
##  3 <NA>        El Salvador     0     0     0     1     0     0     0     0     0
##  4 Alaska      <NA>            9     2     6     0     0     0     0     0     0
##  5 <NA>        Russia          0     0     6     0     0     0     0     0     0
##  6 <NA>        Ukraine         9     2     0     0     0     0     0     0     0
##  7 Arizona     <NA>           26    47    71    73   110     0     0     0     0
##  8 <NA>        Afghanistan     0     1    14    10    17     0     0     0     0
##  9 <NA>        Algeria         0     0     0     0     1     0     0     0     0
## 10 <NA>        Bhutan          2     3     8     2     0     0     0     0     0
## # ℹ 490 more rows
## # ℹ 7 more variables: Jul <dbl>, Aug <dbl>, Sep <dbl>, Cases <dbl>, Inds <dbl>,
## #   State <dbl>, U.S. <dbl>

Fill down the Destination:

ref_clean <-
    fill(ref, Destination)
ref_clean
## # A tibble: 500 × 18
##    Destination Nationality   Oct   Nov   Dec   Jan   Feb   Mar   Apr   May   Jun
##    <chr>       <chr>       <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
##  1 Alabama     <NA>            0     0     3     1     4     0     0     0     0
##  2 Alabama     Dem. Rep. …     0     0     3     0     4     0     0     0     0
##  3 Alabama     El Salvador     0     0     0     1     0     0     0     0     0
##  4 Alaska      <NA>            9     2     6     0     0     0     0     0     0
##  5 Alaska      Russia          0     0     6     0     0     0     0     0     0
##  6 Alaska      Ukraine         9     2     0     0     0     0     0     0     0
##  7 Arizona     <NA>           26    47    71    73   110     0     0     0     0
##  8 Arizona     Afghanistan     0     1    14    10    17     0     0     0     0
##  9 Arizona     Algeria         0     0     0     0     1     0     0     0     0
## 10 Arizona     Bhutan          2     3     8     2     0     0     0     0     0
## # ℹ 490 more rows
## # ℹ 7 more variables: Jul <dbl>, Aug <dbl>, Sep <dbl>, Cases <dbl>, Inds <dbl>,
## #   State <dbl>, U.S. <dbl>

Drop the totals:

ref_clean <-
    fill(ref, Destination) %>%
    filter(! is.na(Nationality))
ref_clean
## # A tibble: 451 × 18
##    Destination Nationality   Oct   Nov   Dec   Jan   Feb   Mar   Apr   May   Jun
##    <chr>       <chr>       <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
##  1 Alabama     Dem. Rep. …     0     0     3     0     4     0     0     0     0
##  2 Alabama     El Salvador     0     0     0     1     0     0     0     0     0
##  3 Alaska      Russia          0     0     6     0     0     0     0     0     0
##  4 Alaska      Ukraine         9     2     0     0     0     0     0     0     0
##  5 Arizona     Afghanistan     0     1    14    10    17     0     0     0     0
##  6 Arizona     Algeria         0     0     0     0     1     0     0     0     0
##  7 Arizona     Bhutan          2     3     8     2     0     0     0     0     0
##  8 Arizona     Burma           6     3     7     1     9     0     0     0     0
##  9 Arizona     Burundi         0     0     0     5     7     0     0     0     0
## 10 Arizona     Colombia        0     0     2     0     0     0     0     0     0
## # ℹ 441 more rows
## # ℹ 7 more variables: Jul <dbl>, Aug <dbl>, Sep <dbl>, Cases <dbl>, Inds <dbl>,
## #   State <dbl>, U.S. <dbl>

Pivot to a tidier form with one row per month:

ref_clean <-
    fill(ref, Destination) %>%
    filter(! is.na(Nationality)) %>%
    pivot_longer(
        Oct : Sep,
        names_to = "month",
        values_to = "count")
ref_clean
## # A tibble: 5,412 × 8
##    Destination Nationality     Cases  Inds State     U.S. month count
##    <chr>       <chr>           <dbl> <dbl> <dbl>    <dbl> <chr> <dbl>
##  1 Alabama     Dem. Rep. Congo     2     7 0.875 0.000811 Oct       0
##  2 Alabama     Dem. Rep. Congo     2     7 0.875 0.000811 Nov       0
##  3 Alabama     Dem. Rep. Congo     2     7 0.875 0.000811 Dec       3
##  4 Alabama     Dem. Rep. Congo     2     7 0.875 0.000811 Jan       0
##  5 Alabama     Dem. Rep. Congo     2     7 0.875 0.000811 Feb       4
##  6 Alabama     Dem. Rep. Congo     2     7 0.875 0.000811 Mar       0
##  7 Alabama     Dem. Rep. Congo     2     7 0.875 0.000811 Apr       0
##  8 Alabama     Dem. Rep. Congo     2     7 0.875 0.000811 May       0
##  9 Alabama     Dem. Rep. Congo     2     7 0.875 0.000811 Jun       0
## 10 Alabama     Dem. Rep. Congo     2     7 0.875 0.000811 Jul       0
## # ℹ 5,402 more rows

Top 5 destination states:

group_by(ref_clean, Destination) %>%
    summarize(count = sum(count)) %>%
    ungroup() %>%
    slice_max(count, n = 5)
## # A tibble: 5 × 2
##   Destination count
##   <chr>       <dbl>
## 1 Ohio          750
## 2 Texas         602
## 3 Washington    562
## 4 California    545
## 5 New York      540

Top 5 origin nationalities:

group_by(ref_clean, Nationality) %>%
    summarize(count = sum(count)) %>%
    ungroup() %>%
    slice_max(count, n = 5)
## # A tibble: 5 × 2
##   Nationality     count
##   <chr>           <dbl>
## 1 Dem. Rep. Congo  1968
## 2 Bhutan           1884
## 3 Burma            1240
## 4 Ukraine          1021
## 5 Eritrea           648

Completing Missing Factor Level Combinations

This example is based on a blog post using data from the Significant Earthquake Database.

The data used is a selection for North America and Hawaii.

Reading in the data:

if (! file.exists("earthquakes.tsv"))
    download.file("http://homepage.stat.uiowa.edu/~luke/data/earthquakes.tsv",
                  "earthquakes.tsv")
eq <- read.table("earthquakes.tsv",
                 header = TRUE,
                 sep = "\t")
eq <- as_tibble(eq)
select(eq, STATE, everything())
## # A tibble: 254 × 47
##    STATE   I_D FLAG_TSUNAMI  YEAR MONTH   DAY  HOUR MINUTE SECOND FOCAL_DEPTH
##    <chr> <int> <chr>        <int> <int> <int> <int>  <int>  <dbl>       <int>
##  1 HI     6697 Tsu           1500    NA    NA    NA     NA     NA          NA
##  2 MA     6013 Tsu           1668     4    13    NA     NA     NA          NA
##  3 OR     9954 Tsu           1700     1    27     5      0     NA          NA
##  4 MA     5828 Tsu           1755    11    18     9     11     35          NA
##  5 AK     5926 Tsu           1788     7    21    NA     NA     NA          NA
##  6 AK     5927 Tsu           1788     8     6    NA     NA     NA          NA
##  7 AK     7057 Tsu           1792    NA    NA    NA     NA     NA          NA
##  8 CA     5857 Tsu           1806     3    25     8     NA     NA          NA
##  9 AR     1594 Tsu           1811    12    16     8     15     NA          NA
## 10 AR     7058 Tsu           1811    12    16    14     15     NA          NA
## # ℹ 244 more rows
## # ℹ 37 more variables: EQ_PRIMARY <dbl>, EQ_MAG_MW <dbl>, EQ_MAG_MS <dbl>,
## #   EQ_MAG_MB <dbl>, EQ_MAG_ML <dbl>, EQ_MAG_MFA <dbl>, EQ_MAG_UNK <dbl>,
## #   INTENSITY <int>, COUNTRY <chr>, LOCATION_NAME <chr>, LATITUDE <dbl>,
## #   LONGITUDE <dbl>, REGION_CODE <int>, DEATHS <int>, DEATHS_DESCRIPTION <int>,
## #   MISSING <lgl>, MISSING_DESCRIPTION <lgl>, INJURIES <int>,
## #   INJURIES_DESCRIPTION <int>, DAMAGE_MILLIONS_DOLLARS <dbl>, …

Some STATE entries are blank, but the location name identifies the state:

filter(eq, STATE == "") %>%
    select(STATE, LOCATION_NAME)
## # A tibble: 9 × 2
##   STATE LOCATION_NAME                        
##   <chr> <chr>                                
## 1 ""    MONTANA:  CLARKSTON VALLEY           
## 2 ""    ILLINOIS:  WEST SALEM                
## 3 ""    NEVADA:  FALLON                      
## 4 ""    CALIFORNIA:  OCOTILLO                
## 5 ""    COLORADO:  PAONIA                    
## 6 ""    ALASKA: ALEUTIAN ISLANDS: FOX ISLANDS
## 7 ""    OKLAHOMA:  SPARKS                    
## 8 ""    OKLAHOMA:  SPARKS, PRAGUE            
## 9 ""    TEXAS:  WEST

One way to fix this:

badidx <- which(eq$STATE == "")
badstate <- sub(":.*", "",
                eq$LOCATION_NAME[badidx])
eq$STATE[badidx] <-
    state.abb[match(tolower(badstate),
                    tolower(state.name))]

Add full state names as a factor:

states <- data.frame(STATE = state.abb,
                     STATE_NAME = state.name)
eq <- left_join(eq, states, "STATE")
eq <- mutate(eq,
             STATE_NAME = factor(STATE_NAME,
                                 state.name))

Look at total number of earthquakes by state:

tbl <- count(eq, STATE_NAME)
head(tbl, 6)
## # A tibble: 6 × 2
##   STATE_NAME      n
##   <fct>       <int>
## 1 Alabama         1
## 2 Alaska         78
## 3 Arkansas        2
## 4 California    100
## 5 Colorado        2
## 6 Connecticut     1
library(forcats)
pb <- ggplot(tbl, aes(y = fct_rev(STATE_NAME), x = n)) +
    geom_col() +
    thm +
    theme(axis.title.y = element_blank())
ps <- ggplot(tbl, aes(y = fct_rev(STATE_NAME), x = n)) +
    geom_point() +
    geom_segment(aes(xend = 0, yend = fct_rev(STATE_NAME))) +
    thm +
    theme(axis.title.y = element_blank())
pb | ps

Many states, including Iowa, are missing.

Their counts should be zero.

This often impacts visualizations.

complete() can be used to fill in the zero values.

tbl <-
    count(eq, STATE_NAME) %>%
    complete(STATE_NAME, fill = list(n = 0))
tbl
## # A tibble: 50 × 2
##    STATE_NAME      n
##    <fct>       <int>
##  1 Alabama         1
##  2 Alaska         78
##  3 Arizona         0
##  4 Arkansas        2
##  5 California    100
##  6 Colorado        2
##  7 Connecticut     1
##  8 Delaware        0
##  9 Florida         0
## 10 Georgia         0
## # ℹ 40 more rows
thm_ytxt_sm <- theme(axis.text.y = element_text(size = 10))
(pb %+% tbl + thm_ytxt_sm) | (ps %+% tbl + thm_ytxt_sm)

Reading

Chapter Data transformation of R for Data Science.

Chapter Tidy data of R for Data Science.

Interactive Tutorial

An interactive learnr tutorial for these notes is available.

You can run the tutorial with

STAT4580::runTutorial("dplyr")

You can install the current version of the STAT4580 package with

remotes::install_gitlab("luke-tierney/STAT4580")

You may need to install the remotes package from CRAN first.

Exercises

  1. To bring in dplyr and the mpg data, start by evaluating

    library(dplyr)
    library(ggplot2)

    The select function allows variables to be specified in a variety of ways. Which of the following does not produce a data frame with only the variables manufacturer, model, cty, hwy?

    1. select(mpg, 1:2, 7:8)
    2. select(mpg, starts_with("m"), ends_with("y"))
    3. select(mpg, 1:2, cty : hwy)
    4. select(mpg, -(displ : drv), -(fl : class))
  2. Consider the code

    library(dplyr)
    library(ggplot2)
    filter(mpg, ---) %>% nrow()

    Which of the replacements for --- computes the number of Ford vehicles with more than 6 cylinders in the mpg table?

    1. model == "ford", cyl <= 6
    2. manufacturer == "ford" | cyl > 6
    3. manufacturer == "ford", cyl > 6
    4. manufacturer == "ford", cylinders > 6
  3. In the 2013 NYC flights data provided by the nycflights13 package how many flights were there to Des Moines (FAA code DSM) from NYC in the first three months of 2013?

    1. 13
    2. 98
    3. 64
    4. 77
  4. To bring in dplyr and the mpg data, start by evaluating

    library(dplyr)
    library(ggplot2)

    Which of the following sorts the rows for mpg by increasing cyl value and, within each cyl value sorts the rows from largest to smallest hwy value.

    1. arrange(mpg, desc(hwy), cyl)
    2. arrange(mpg, cyl, desc(hwy))
    3. arrange(mpg, desc(cyl), hwy)
    4. arrange(mpg, cyl, hwy)
  5. To bring in dplyr and the flights data, start by evaluating

    library(dplyr)
    library(nycflights13)

    The dep_time variable in the flights data set from the nycflights13 package is convenient to look at (529 means 5:29 AM and 22:12 means 10:12 PM), but hard to compute with. Which of the following adds variables dep_hour and dep_min containing hour and minute of the departure time?

    1. mutate(flights, dep_hour = dep_time %/% 60, dep_min = dep_time %% 60)
    2. mutate(flights, dep_hour = dep_time %/% 100, dep_min = dep_time %% 100)
    3. mutate(flights, dep_hour = dep_time / 100, dep_min = dep_time - dep_hour)
    4. mutate(flights, dep_hour = dep_time %/% 60, dep_min = dep_hour %% 60)
  6. Using the gapminder data set, the following code computes population totals for each continent and each year:

    library(dplyr)
    library(gapminder)
    cpops <- group_by(gapminder, continent, year) %>%
        summarize(pop = sum(pop)) %>%
        ungroup()

    To produce a plot to compare population growth over the years for the continents it is useful to standardize the population data for each continent, for example by dividing the population values by the average population size for each continent. One way to do this is with a grouped mutate. The first line of your result should be

    head(cpops_std, 1)
    ## # A tibble: 1 × 4
    ##   continent  year       pop stdpop
    ##   <fct>     <int>     <dbl>  <dbl>
    ## 1 Africa     1952 237640501  0.461
    library(ggplot2)
    ggplot(cpops_std, aes(x = year, y = stdpop, color = continent)) +
        geom_line()

    Which of the following produces the correct result:

    1. cpops_std <- group_by(cpops, continent) %>% mutate(stdpop = pop / mean(year)) %>% ungroup()
    2. cpops_std <- group_by(cpops, year) %>% mutate(stdpop = pop / mean(pop)) %>% ungroup()
    3. cpops_std <- group_by(cpops, continent) %>% mutate(stdpop = pop / mean(pop)) %>% ungroup()
    4. cpops_std <- group_by(cpops, year) %>% mutate(stdpop = pop / mean(year)) %>% ungroup()
  7. Another approach to the previous exercise first creates a table of population averages with

    cpops_avg <- group_by(cpops, continent) %>%
        summarize(avg_pop = mean(pop))

    Then use a left join to add avg_pop to the cpops table, followed by an ordinary mutate step:

    left_join(---) %>%
        mutate(stdpop = pop / avg_pop)

    Which is the correct replacement for ---?

    1. cpops_avg, cpops, "continent"
    2. cpops, cpops_avg, "continent"
    3. cpops, cpops_avg, "year"
    4. cpops_avg, cpops, "year"
LS0tCnRpdGxlOiAiQSBHcmFtbWFyIG9mIERhdGEgTWFuaXB1bGF0aW9uIgpvdXRwdXQ6CiAgaHRtbF9kb2N1bWVudDoKICAgIHRvYzogeWVzCiAgICBjb2RlX2Rvd25sb2FkOiB0cnVlCi0tLQoKPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBocmVmPSJzdGF0NDU4MC5jc3MiIHR5cGU9InRleHQvY3NzIiAvPgo8c3R5bGUgdHlwZT0idGV4dC9jc3MiPiAucmVtYXJrLWNvZGUgeyBmb250LXNpemU6IDg1JTsgfSA8L3N0eWxlPgoKYGBge3Igc2V0dXAsIGluY2x1ZGUgPSBGQUxTRSwgbWVzc2FnZSA9IEZBTFNFfQpzb3VyY2UoaGVyZTo6aGVyZSgic2V0dXAuUiIpKQprbml0cjo6b3B0c19jaHVuayRzZXQoY29sbGFwc2UgPSBUUlVFLCBtZXNzYWdlID0gRkFMU0UsCiAgICAgICAgICAgICAgICAgICAgICBmaWcuaGVpZ2h0ID0gNSwgZmlnLndpZHRoID0gNiwgZmlnLmFsaWduID0gImNlbnRlciIpCgpzZXQuc2VlZCgxMjM0NSkKbGlicmFyeShkcGx5cikKbGlicmFyeShnZ3Bsb3QyKQpsaWJyYXJ5KGxhdHRpY2UpCmBgYAoKCiMjIEJhY2tncm91bmQKClRoZSBgZHBseXJgIHBhY2thZ2UgcHJvdmlkZXMgYSBsYW5ndWFnZSwgb3IgZ3JhbW1hciwgZm9yIGRhdGEKbWFuaXB1bGF0aW9uLgoKVGhlIGxhbmd1YWdlIGNvbnRhaW5zIGEgbnVtYmVyIG9mIF92ZXJic18gdGhhdCBvcGVyYXRlIG9uIHRhYmxlcyBpbgpkYXRhIGZyYW1lcy4KClRoZSBtb3N0IGNvbW1vbmx5IHVzZWQgdmVyYnMgb3BlcmF0ZSBvbiBhIHNpbmdsZSBkYXRhIGZyYW1lOgoKKiBbYHNlbGVjdCgpYF0oaHR0cHM6Ly9kcGx5ci50aWR5dmVyc2Uub3JnL3JlZmVyZW5jZS9zZWxlY3QuaHRtbCk6CiAgcGljayB2YXJpYWJsZXMgYnkgdGhlaXIgbmFtZXMKCiogW2BmaWx0ZXIoKWBdKGh0dHBzOi8vZHBseXIudGlkeXZlcnNlLm9yZy9yZWZlcmVuY2UvZmlsdGVyLmh0bWwpOgogIGNob29zZSByb3dzIHRoYXQgc2F0aXNmeSBzb21lIGNyaXRlcmlhCgoqIFtgbXV0YXRlKClgXShodHRwczovL2RwbHlyLnRpZHl2ZXJzZS5vcmcvcmVmZXJlbmNlL211dGF0ZS5odG1sKToKICBjcmVhdGUgdHJhbnNmb3JtZWQgb3IgZGVyaXZlZCB2YXJpYWJsZXMKCiogW2BhcnJhbmdlKClgXShodHRwczovL2RwbHlyLnRpZHl2ZXJzZS5vcmcvcmVmZXJlbmNlL2FycmFuZ2UuaHRtbCk6CiAgcmVvcmRlciB0aGUgcm93cwoKKiBbYHN1bW1hcml6ZSgpYF0oaHR0cHM6Ly9kcGx5ci50aWR5dmVyc2Uub3JnL3JlZmVyZW5jZS9zdW1tYXJpc2UuaHRtbCk6CiAgY29sbGFwc2Ugcm93cyBkb3duIHRvIHN1bW1hcmllcwoKVGhlcmUgYXJlIGFsc28gYSBudW1iZXIgb2YgYGpvaW5gIHZlcmJzIHRoYXQgbWVyZ2Ugc2V2ZXJhbCBkYXRhIGZyYW1lcwppbnRvIG9uZS4KClRoZSBgdGlkeXJgIHBhY2thZ2UgcHJvdmlkZXMgYWRkaXRpb25hbCB2ZXJicywgc3VjaCBhcyBgcGl2b3RfbG9uZ2VyYAphbmQgYHBpdm90X3dpZGVyYCBmb3IgcmVzaGFwaW5nIGRhdGEgZnJhbWVzLgoKVGhlIHNpbmdsZSB0YWJsZSB2ZXJicyBjYW4gYWxzbyBiZSB1c2VkIHdpdGgKW2Bncm91cF9ieSgpYF0oaHR0cHM6Ly9kcGx5ci50aWR5dmVyc2Uub3JnL3JlZmVyZW5jZS9ncm91cF9ieS5odG1sKSBhbmQKW2B1bmdyb3VwKClgXShodHRwczovL2RwbHlyLnRpZHl2ZXJzZS5vcmcvcmVmZXJlbmNlL2dyb3VwX2J5Lmh0bWwpIHRvCndvcmsgYSBncm91cCBhdCBhIHRpbWUgaW5zdGVhZCBvZiBhcHBseWluZyB0byB0aGUgZW50aXJlIGRhdGEgZnJhbWUuCgpUaGUgZGVzaWduIG9mIGBkcGx5cmAgaXMgc3Ryb25nbHkgbW90aXZhdGVkIGJ5IFNRTC4KCiogYGRwbHlyYCBjYW4gYWxzbyBiZSB1c2VkIHRvIG9wZXJhdGUgb24KICBbdGFibGVzIHN0b3JlZCBpbiBkYXRhIGJhc2VzLl0oaHR0cHM6Ly9kYnBseXIudGlkeXZlcnNlLm9yZy9pbmRleC5odG1sKQoKU29tZSBkYXRhIHNldHMgZm9yIGlsbHVzdHJhdGlvbjoKCiogRVBBIHZlaGljbGUgZGF0YSB1c2VkIGluIEhXNApgYGB7ciwgbWVzc2FnZSA9IEZBTFNFLCBjbGFzcy5zb3VyY2UgPSAiZm9sZC1oaWRlIn0KbGlicmFyeShyZWFkcikKaWYgKCEgZmlsZS5leGlzdHMoInZlaGljbGVzLmNzdi56aXAiKSkKICAgIGRvd25sb2FkLmZpbGUoImh0dHA6Ly93d3cuc3RhdC51aW93YS5lZHUvfmx1a2UvZGF0YS92ZWhpY2xlcy5jc3YuemlwIiwKICAgICAgICAgICAgICAgICAgInZlaGljbGVzLmNzdi56aXAiKQpuZXdtcGcgPC0gcmVhZF9jc3YoInZlaGljbGVzLmNzdi56aXAiLCBndWVzc19tYXggPSAxMDAwMDApCmBgYAoKKiBUaGUgYG55Y2ZsaWdodHMxM2AgcGFja2FnZSBwcm92aWRlcyBkYXRhIG9uIGFsbCBmbGlnaHRzCiAgb3JpZ2luYXRpbmcgZnJvbSBvbmUgb2YgdGhlIHRocmVlIG1haW4gTmV3IFlvcmsgQ2l0eSBhaXJwb3J0cyBpbgogIDIwMTMgYW5kIGhlYWRpbmcgdG8gYWlycG9ydHMgd2l0aGluIHRoZSBVUy4KYGBge3IsIGNsYXNzLnNvdXJjZSA9ICJmb2xkLWhpZGUifQpsaWJyYXJ5KG55Y2ZsaWdodHMxMykKYGBgCgoqIFRoZSBgc3Rvcm1zYCBkYXRhIGZyYW1lLCBpbmNsdWRlZCBpbiBgZHBseXJgIHdpdGggZGF0YSBvbgogIGh1cnJpY2FuZXMgYmV0d2VlbiAxOTc1IGFuZCAyMDE1LgoKVGhlIGJhc2ljIHRyYW5zZm9ybWF0aW9uIHZlcmJzIGFyZSBkZXNjcmliZWQgaW4gdGhlCltEYXRhIFRyYW5zZm9ybWF0aW9uIGNoYXB0ZXJdKGh0dHBzOi8vcjRkcy5oYWQuY28ubnovdHJhbnNmb3JtLmh0bWwpIG9mCltSIGZvciBEYXRhIFNjaWVuY2VdKGh0dHBzOi8vcjRkcy5oYWQuY28ubnovKS4KClV0aWxpdGllcyBpbiBgdGlkeXJgIGFyZSBkZXNjcmliZWQgaW4gdGhlIFtUaWR5IERhdGEKY2hhcHRlcl0oaHR0cHM6Ly9yNGRzLmhhZC5jby5uei90aWR5LWRhdGEuaHRtbCkuCgoKIyMgU2VsZWN0aW5nIFZhcmlhYmxlcwoKRGF0YSBzZXRzIG9mdGVuIGNvbnRhaW4gaHVuZHJlZHMgb2YgZXZlbiB0aG91c2FuZHMgb2YgdmFyaWFibGVzLgoKQSB1c2VmdWwgZmlyc3Qgc3RlcCBpcyB0byBzZWxlY3QgYSBncm91cCBvZiBpbnRlcmVzdGluZyB2YXJpYWJsZXMuCgpBIHJlYXNvbmFibGUgc2VsZWN0aW9uIG9mIHRoZSBFUEEgdmVoaWNsZSB2YXJpYWJsZXM6CgpgYGB7cn0KbmV3bXBnMSA8LSBzZWxlY3QobmV3bXBnLAogICAgICAgICAgICAgICAgICBtYWtlLCBtb2RlbCwgeWVhciwKICAgICAgICAgICAgICAgICAgY3R5ID0gY2l0eTA4LCBod3kgPSBoaWdod2F5MDgsCiAgICAgICAgICAgICAgICAgIHRyYW5zID0gdHJhbnksIGN5bCA9IGN5bGluZGVycywKICAgICAgICAgICAgICAgICAgZnVlbCA9IGZ1ZWxUeXBlMSwgZGlzcGwpCmhlYWQobmV3bXBnMSwgMikKYGBgCgo8ZGl2IGNsYXNzID0gImFsZXJ0Ij4KVmFyaWFibGVzIGNhbiBiZSBnaXZlbiBuZXcgbmFtZXMgaW4gYSBgc2VsZWN0YCBjYWxsOyB5b3UgY2FuIGFsc28KcmVuYW1lIHRoZW0gbGF0ZXIgd2l0aCBgcmVuYW1lYC4KPC9kaXY+CgpTb21lIHZhcmlhdGlvbnMgKHRoZSBkb2N1bWVudGF0aW9uIGZvcgpbYHNlbGVjdCgpYF0oaHR0cHM6Ly9kcGx5ci50aWR5dmVyc2Uub3JnL3JlZmVyZW5jZS9zZWxlY3QuaHRtbCkgZ2l2ZXMKZnVsbCBkZXRhaWxzKToKCmBgYHtyfQpzZWxlY3QobmV3bXBnMSwgeWVhciA6IHRyYW5zKSAlPiUgaGVhZCgyKQpgYGAKCmBgYHtyfQpzZWxlY3QobmV3bXBnMSwgLXllYXIsIC10cmFucykgJT4lIGhlYWQoMikKYGBgCgpgYGB7cn0Kc2VsZWN0KG5ld21wZzEsIC0gKHllYXIgOiB0cmFucykpICU+JSBoZWFkKDIpCmBgYAoKTnVtZXJpY2FsIGNvbHVtbiBzcGVjaWZpY2F0aW9ucyBjYW4gYWxzbyBiZSB1c2VkOgoKYGBge3J9CnNlbGVjdChuZXdtcGcxLCAxLCAzKSAlPiUgaGVhZCgyKQpgYGAKCmBgYHtyfQpzZWxlY3QobmV3bXBnMSwgMSA6IDMpICU+JSBoZWFkKDIpCmBgYAoKYGBge3J9CnNlbGVjdChuZXdtcGcxLCAtICgxIDogMykpICU+JSBoZWFkKDIpCmBgYAoKU29tZSBfaGVscGVyIGZ1bmN0aW9uc18gY2FuIGFsc28gYmUgdXNlZCBpbiB0aGUgY29sdW1uIHNwZWNpZmljYXRpb25zLgoKVGhlIG1vc3QgdXNlZnVsIG9uZXMgYXJlIGBzdGFydHNfd2l0aGAsIGBlbmRzX3dpdGhgLCBhbmQgYGNvbnRhaW5zYC4KCmBgYHtyfQpzZWxlY3QobmV3bXBnLCBzdGFydHNfd2l0aCgiZnVlbCIpKSAlPiUgaGVhZCgyKQpgYGAKCmBgYHtyfQpzZWxlY3QobmV3bXBnLCBjb250YWlucygiY2l0eSIpKSAlPiUgaGVhZCgyKQpgYGAKCjxkaXYgY2xhc3MgPSAiYWxlcnQiPgpCeSBkZWZhdWx0IHRoZXNlIGhlbHBlcnMgaWdub3JlIGNhc2UuCjwvZGl2PgoKVGhlc2UgY2FuIGFsc28gYmUgcHJlY2VkZWQgYnkgYSBtaW51cyBtb2RpZmllciB0byBvbWl0IGNvbHVtbnM6CgpgYGB7cn0Kc2VsZWN0KG5ld21wZzEsIC1jb250YWlucygibSIpKSAlPiUgaGVhZCgyKQpgYGAKCmBjb250YWluc2AgcmVxdWlyZXMgYSBsaXRlcmFsIHN0cmluZzsgYG1hdGNoZXNgIGlzIGFuYWxvZ291cyBidXQgaXRzCmFyZ3VtZW50IGlzIGludGVycHJldGVkIGFzIGEgW19yZWd1bGFyIGV4cHJlc3Npb25fXShyZWdleC5odG1sKQpwYXR0ZXJuIHRvIGJlIG1hdGNoZWQuCgpgYGB7cn0Kc2VsZWN0KG5ld21wZywgbWF0Y2hlcygiXltGZl11ZWwiKSkgJT4lIGhlYWQoMikKYGBgCgpTb21ldGltZXMgaXQgaXMgdXNlZnVsIHRvIG1vdmUgYSBmZXcgY29sdW1ucyB0byB0aGUgZnJvbnQ7IHRoZSBgZXZlcnl0aGluZ2AKaGVscGVyIGNhbiBiZSB1c2VkIGZvciB0aGF0LgoKVGhlIG9yaWdpbmFsIE5ZQyBgZmxpZ2h0c2AgdGFibGU6CgpgYGB7cn0KaGVhZChmbGlnaHRzLCAyKQpgYGAKCk1vdmluZyBgYWlyX3RpbWVgIHRvIHRoZSBmcm9udDoKCmBgYHtyfQpzZWxlY3QoZmxpZ2h0cywgYWlyX3RpbWUsIGV2ZXJ5dGhpbmcoKSkgJT4lIGhlYWQoMikKYGBgCgpUaGUgYHdoZXJlYCBoZWxwZXIgZnVuY3Rpb24gYWxsb3dzIGNvbHVtbnMgdG8gYmUgc2VsZWN0ZWQgYmFzZWQgb24gYQpwcmVkaWNhdGUgYXBwbGllZCB0byB0aGUgZnVsbCBjb2x1bW46CgpgYGB7cn0Kc2VsZWN0KG5ld21wZzEsIHdoZXJlKGFueU5BKSkgJT4lIGhlYWQoMikKYGBgCgpgYGB7cn0Kc2VsZWN0KGZsaWdodHMsIHdoZXJlKGFueU5BKSkgJT4lIGhlYWQoMikKYGBgCgo8ZGl2IGNsYXNzID0gImFsZXJ0Ij4KSWYgdGhlcmUgYXJlIG1pc3NpbmcgdmFsdWVzIGluIHlvdXIgZGF0YSBpdCBpcyB1c3VhbGx5IGEgZ29vZCBpZGVhIHRvCnJldmlldyB0aGVtIHRvIG1ha2Ugc3VyZSB5b3UgdW5kZXJzdGFuZCB3aGF0IHRoZXkgbWVhbi4KPC9kaXY+CgpDb2x1bW5zIGNhbiBhbHNvIGJlIHNlbGVjdGVkIHVzaW5nIGxpc3Qgb3BlcmF0aW9uczoKCmBgYHtyfQp2YXJzIDwtIGMoIm1ha2UiLCAibW9kZWwiLCAieWVhciIsICJjaXR5MDgiLCAidHJhbnkiLCAiY3lsaW5kZXJzIiwKICAgICAgICAgICJmdWVsVHlwZTEiLCAiZGlzcGwiKQpuZXdtcGdbdmFyc10gJT4lIGhlYWQoMikKYGBgCgoKIyMgRmlsdGVyaW5nIFJvd3MKClRoZSBgZmlsdGVyYCBmdW5jdGlvbiBwaWNrcyBvdXQgdGhlIHN1YnNldCBvZiByb3dzIHRoYXQgc2F0aXNmeSBvbmUgb3IKbW9yZSBjb25kaXRpb25zLgoKQWxsIGNhcnMgd2l0aCBjaXR5IE1QRyBhdCBvciBhYm92ZSAxMzAgYW5kIG1vZGVsIHllYXIgMjAxODoKCmBgYHtyfQpmaWx0ZXIobmV3bXBnMSwgY3R5ID49IDEzMCwgeWVhciA9PSAyMDE4KQpgYGAKCkFsbCBmbGlnaHRzIGZyb20gTmV3IFlvcmsgdG8gRGVzIE1vaW5lcyBvbiBKYW51YXJ5IDEsIDIwMTM6CgpgYGB7cn0KZmlsdGVyKGZsaWdodHMsIGRheSA9PSAxLCBtb250aCA9PSAxLCBkZXN0ID09ICJEU00iKSAlPiUKICAgIHNlbGVjdCh5ZWFyIDogZGVwX3RpbWUsIG9yaWdpbikKYGBgCgoKIyMjIENvbXBhcmlzb25zCgpUaGUgYmFzaWMgY29tcGFyaXNvbiBvcGVyYXRvcnMgYXJlCgoqIGA9PWAsIGAhPWAKKiBgPGAsIGA8PWAKKiBgPmAsIGA+PWAKCjxkaXYgY2xhc3MgPSAiYWxlcnQiPgpCZSBzdXJlIHRvIHVzZSBgPT1gLCBub3QgYSBzaW5nbGUgYD1gIGNoYXJhY3Rlci4KPC9kaXY+Cgo8ZGl2IGNsYXNzID0gImFsZXJ0Ij4KQmUgY2FyZWZ1bCBhYm91dCBmbG9hdGluZyBwb2ludCBlcXVhbGl0eSB0ZXN0czoKCmBgYHtyfQp4IDwtIDEgLSAwLjkKeCA9PSAwLjEKbmVhcih4LCAwLjEpCmBgYAo8L2Rpdj4KCjxkaXYgY2xhc3MgPSAiYWxlcnQiPgpFcXVhbGl0eSBjb21wYXJpc29ucyBjYW4gYmUgdXNlZCBvbiBjaGFyYWN0ZXIgdmVjdG9ycyBhbmQgZmFjdG9ycy4KCk9yZGVyIGNvbXBhcmlzb25zIGNhbiBiZSB1c2VkIG9uIGNoYXJhY3RlciB2ZWN0b3JzLCBidXQgbWF5IHByb2R1Y2UKc3VycHJpc2luZyByZXN1bHRzLCBvciByZXN1bHRzIHRoYXQgdmFyeSB3aXRoIGRpZmZlcmVudCBsb2NhbGUKc2V0dGluZ3MuCjwvZGl2PgoKYCVpbiVgIGNhbiBiZSB1c2VkIHRvIG1hdGNoIGFnYWluc3Qgb25lIG9mIHNldmVyYWwgb3B0aW9uczoKCmBgYHtyfQpmaWx0ZXIoZmxpZ2h0cywgZGF5ICVpbiUgMSA6IDIsIG1vbnRoID09IDEsIGRlc3QgPT0gIkRTTSIpICU+JQogICAgc2VsZWN0KHllYXIgOiBkZXBfdGltZSwgb3JpZ2luKQpgYGAKCgojIyMgTG9naWNhbCBPcGVyYXRpb25zCgpUaGUgYmFzaWMgbG9naWNhbCBvcGVyYXRpb25zIG9uIHZlY3RvcnMgYXJlCgoqIGAmYCAtLSBhbmQKKiBgfGAgLS0gb3IKKiBgIWAgLS0gbm90CiogYHhvcmAgLS0gZXhjbHVzaXZlIG9yCgpUcnV0aCB0YWJsZXM6Cgo8IS0tICMgbm9saW50IHN0YXJ0IC0tPgoKYGBge3J9CnggPC0gYyhUUlVFLCBUUlVFLCAgRkFMU0UsIEZBTFNFKQp5IDwtIGMoVFJVRSwgRkFMU0UsIFRSVUUsICBGQUxTRSkKISB4CnggJiB5CnggfCB5Cnhvcih4LCB5KQpgYGAKPCEtLSAjIG5vbGludCBlbmQgLS0+Cgo8ZGl2IGNsYXNzID0gImFsZXJ0Ij4KTWFrZSBzdXJlIG5vdCB0byBjb25mdXNlIHRoZSB2ZWN0b3JpemVkIGxvZ2ljYWwgb3BlcmF0b3JzIGAmYCBhbmQgYHxgCndpdGggdGhlIHNjYWxhciBmbG93IGNvbnRyb2wgb3BlcmF0b3JzIGAmJmAgYW5kIGB8fGAuCjwvZGl2PgoKVGhlIHByZXZpb3VzIGBmbGlnaHRzYCBleGFtcGxlIGNhbiBhbHNvIGJlIHdyaXR0ZW4gYXMKCmBgYHtyfQpmaWx0ZXIoZmxpZ2h0cywgZGF5ID09IDEgfCBkYXkgPT0gMiwgbW9udGggPT0gMSwgZGVzdCA9PSAiRFNNIikgJT4lCiAgICBzZWxlY3QoeWVhciA6IGRlcF90aW1lLCBvcmlnaW4pCmBgYAoKSXQgY2FuIGFsc28gYmUgd3JpdHRlbiBhcwoKYGBge3J9CmZpbHRlcihmbGlnaHRzLCAoZGF5ID09IDEgfCBkYXkgPT0gMikgJiBtb250aCA9PSAxICYgZGVzdCA9PSAiRFNNIikgJT4lCiAgICBzZWxlY3QoeWVhciA6IGRlcF90aW1lLCBvcmlnaW4pCmBgYAoKCiMjIyBNaXNzaW5nIFZhbHVlcwoKYGZpbHRlcmAgb25seSBrZWVwcyB2YWx1ZXMgY29ycmVzcG9uZGluZyB0byBgVFJVRWAgcHJlZGljYXRlIHZhbHVlczsKYEZBTFNFYCBhbmQgYE5BYCB2YWx1ZXMgYXJlIGRyb3BwZWQuCgpBcml0aG1ldGljIGFuZCBjb21wYXJpc29uIG9wZXJhdG9ycyB3aWxsIGFsd2F5cyBwcm9kdWNlIGBOQWAgdmFsdWVzCmlmIG9uZSBvcGVyYW5kIGlzIGBOQWAuCgo8IS0tICMgbm9saW50IHN0YXJ0IC0tPgpgYGB7cn0KMSArIE5BCk5BIDwgMQpOQSA9PSBOQQpgYGAKPCEtLSAjIG5vbGludCBlbmQgLS0+CgpJbiBhIGZldyBjYXNlcyBhIG5vbi1gTkFgIHJlc3VsdCBjYW4gYmUgZGV0ZXJtaW5lZDoKCmBgYHtyfQpUUlVFIHwgTkEKTkEgJiBGQUxTRQpOQSBeIDAKYGBgCgpgaXMubmFgIGNhbiBiZSB1c2VkIHRvIGFsc28gc2VsZWN0IHJvd3Mgd2l0aCBgTkFgIHZhbHVlcy4KCk5vbi1lbGVjdHJpYyB2ZWhpY2xlcyB3aXRoIHplcm8gb3IgYE5BYCBmb3IgYGRpc3BsYDoKCmBgYHtyfQpmaWx0ZXIobmV3bXBnMSwgZGlzcGwgPT0gMCB8IGlzLm5hKGRpc3BsKSwgZnVlbCAhPSAiRWxlY3RyaWNpdHkiKQpgYGAKCkZsaWdodHMgd2l0aCBgTkFgIGZvciBib3RoIGBkZXBfdGltZWAgYW5kIGBhcnJfdGltZWA6CgpgYGB7cn0KZmlsdGVyKGZsaWdodHMsIGlzLm5hKGRlcF90aW1lKSAmIGlzLm5hKGFycl90aW1lKSkgJT4lCiAgICBoZWFkKDIpICU+JQogICAgc2VsZWN0KHllYXIgOiBhcnJfdGltZSkKYGBgCgpBbiBhbHRlcm5hdGl2ZSBhcHByb2FjaCB0aGF0IGRvZXMgbm90IHVzZSBgZmlsdGVyYCB3b3VsZCBiZQoKYGBge3J9CmlkeCA8LSB3aXRoKGZsaWdodHMsIGlzLm5hKGRlcF90aW1lKSAmIGlzLm5hKGFycl90aW1lKSkKZmxpZ2h0c1tpZHgsIF0gJT4lCiAgICBoZWFkKDIpICU+JQogICAgc2VsZWN0KHllYXIgOiBhcnJfdGltZSkKYGBgCgoKIyMgRXhwbG9yaW5nIHRoZSBFUEEgRGF0YQoKU29ydGluZyBvdXQgZnVlbCB0eXBlczoKCmBgYHtyfQpzZWxlY3QobmV3bXBnLCBjb250YWlucygiZnVlbFR5cGUiKSkgJT4lIHVuaXF1ZSgpCmBgYAoKVmFyaWFibGVzIHdpdGggbWlzc2luZyB2YWx1ZXM6CgpgYGB7cn0Kc2VsZWN0KG5ld21wZywgd2hlcmUoYW55TkEpKSAlPiUgaGVhZCgyKQpgYGAKCkFtb25nIHRoZSByZWR1Y2VkIGRhdGEgc2V0OgoKYGBge3J9CnNlbGVjdChuZXdtcGcxLCB3aGVyZShhbnlOQSkpICU+JSBuYW1lcygpCmBgYAoKVW5pcXVlIG1pc3NpbmcgdmFsdWUgcGF0dGVybnM6CgpgYGB7ciwgZXZhbCA9IEZBTFNFLCBlY2hvID0gRkFMU0V9CiMjIFRoaXMgbWF5IGJlIG1vcmUgcm9idXN0OgppbmNvbXBsZXRlX2Nhc2VzIDwtIGZ1bmN0aW9uKGRhdGEpIGRhdGFbUmVkdWNlKGB8YCwgbGFwcGx5KGRhdGEsIGlzLm5hKSksIF0KYGBgCmBgYHtyfQppbmNvbXBsZXRlX2Nhc2VzIDwtIGZ1bmN0aW9uKGRhdGEpIGRhdGFbISBjb21wbGV0ZS5jYXNlcyhkYXRhKSwgXQoKc2VsZWN0KG5ld21wZzEsIHRyYW5zLCBjeWwsIGRpc3BsLCBmdWVsKSAlPiUKICAgIGluY29tcGxldGVfY2FzZXMoKSAlPiUKICAgIHVuaXF1ZSgpCmBgYAoKQW4gYWx0ZXJuYXRpdmUgdG8gZGVmaW5pbmcgYGluY29tcGxldGVfY2FzZXMoKWA6CgpgYGB7cn0Kc2VsZWN0KG5ld21wZzEsIHRyYW5zLCBjeWwsIGRpc3BsLCBmdWVsKSAlPiUKICAgIGZpbHRlcihpZl9hbnkoZXZlcnl0aGluZygpLCBpcy5uYSkpICU+JQogICAgdW5pcXVlKCkKYGBgCgpMb29raW5nIGF0IHRoZSBjb3VudHM6CgpgYGB7cn0Kc2VsZWN0KG5ld21wZzEsIHRyYW5zLCBjeWwsIGRpc3BsLCBmdWVsKSAlPiUKICAgIGluY29tcGxldGVfY2FzZXMoKSAlPiUKICAgIGNvdW50KHRyYW5zLCBjeWwsIGRpc3BsLCBmdWVsKQpgYGAKCjxkaXYgY2xhc3MgPSAiYWxlcnQiPgpgY291bnQodHJhbnMsIGN5bCwgZGlzcGwsIGZ1ZWwpYCBpcyBlc3NlbnRpYWxseSBhbiBhYmJyZXZpYXRpb24gb2YKCmBgYHtyLCBldmFsID0gRkFMU0V9Cmdyb3VwX2J5KHRyYW5zLCBjeWwsIGRpc3BsLCBmdWVsKSAlPiUKICAgIHN1bW1hcml6ZShuID0gbigpKSAlPiUKICAgIHVuZ3JvdXAoKQpgYGAKPC9kaXY+CgpBbm90aGVyIGFwcHJvYWNoIHRoYXQgYXZvaWRzIHNwZWNpZnlpbmcgdGhlIHZhcmlhYmxlcyB3aXRoIG1pc3NpbmcgdmFsdWVzOgoKYGBge3J9CmluY29tcGxldGVfY2FzZXMobmV3bXBnMSkgJT4lCiAgICBjb3VudChhY3Jvc3Mod2hlcmUoYW55TkEpKSwgZnVlbCkKYGBgCgpDeWxpbmRlcnMgZm9yIGVsZWN0cmljIHZlaGljbGVzOgoKYGBge3J9CmZpbHRlcihuZXdtcGcxLCBmdWVsID09ICJFbGVjdHJpY2l0eSIpICU+JQogICAgY291bnQoaXMubmEoY3lsKSkKYGBgCgpEaXNwbGFjZW1lbnQgZm9yIGVsZWN0cmljIHZlaGljbGVzOgoKYGBge3J9CmZpbHRlcihuZXdtcGcxLCBmdWVsID09ICJFbGVjdHJpY2l0eSIpICU+JQogICAgY291bnQoaXMubmEoZGlzcGwpKQpgYGAKCkVsZWN0cmljIHZlaGljbGVzIHdpdGggbm9uLW1pc3NpbmcgZGlzcGxhY2VtZW50OgoKYGBge3J9CmZpbHRlcihuZXdtcGcxLCBmdWVsID09ICJFbGVjdHJpY2l0eSIsICEgaXMubmEoZGlzcGwpKQpgYGAKClNvbWUgdGhpbmdzIHRvIHRoaW5rIGFib3V0OgoKKiBTaG91bGQgZWxlY3RyaWMgdmVoaWNsZXMgYmUgZHJvcHBlZCBmcm9tIGFuYWx5c2VzIG9yIHZpc3VhbGl6YXRpb25zCiAgaW52b2x2aW5nIGN5bGluZGVycyBvciBkaXNwbGFjZW1lbnQ/CgoqIFNob3VsZCBgY3lsYCBvciBgZGlzcGxgIGJlIHJlY29kZWQsIHBlcmhhcHMgYXMgemVybz8KCiogV2h5IGFyZSBtaWxlcyBwZXIgZ2FsbG9uIF9ub3RfIG1pc3NpbmcsIG9yIGluZmluaXRlLCBmb3IgZWxlY3RyaWMKICB2ZWhpY2xlcz8gVGhlIFtkb2N1bWVudGF0aW9uIGZvciB0aGUKICBkYXRhXShodHRwczovL3d3dy5mdWVsZWNvbm9teS5nb3YvZmVnL3dzL2luZGV4LnNodG1sI3ZlaGljbGUpIGhhcwogIHNvbWUgcG9pbnRlcnMuCgoKIyMgQWRkaW5nIE5ldyBWYXJpYWJsZXMKCmBtdXRhdGUoKWAgY2FuIGJlIHVzZWQgdG8gZGVmaW5lIG5ldyB2YXJpYWJsZXMgb3IgbW9kaWZ5IGV4aXN0aW5nIG9uZXMuCgpOZXcgdmFyaWFibGVzIGFyZSBhZGRlZCBhdCB0aGUgZW5kLgoKTGF0ZXIgdmFyaWFibGVzIGNhbiBiZSBkZWZpbmVkIGluIHRlcm1zIG9mIGVhcmxpZXIgb25lcy4KCjxkaXYgY2xhc3MgPSAiYWxlcnQiPgpgbXV0YXRlKClgIGRvZXMgbm90IG1vZGlmeSB0aGUgb3JpZ2luYWwgZGF0YSBmcmFtZTsgaXQgY3JlYXRlcyBhIG5ldyBvbmUKd2l0aCB0aGUgc3BlY2lmaWVkIG1vZGlmaWNhdGlvbnMuCjwvZGl2PgoKYGBge3J9CmZsIDwtIHNlbGVjdChmbGlnaHRzLCB5ZWFyLCBtb250aCwgZGF5LCBhaXJfdGltZSwgZW5kc193aXRoKCJkZWxheSIpKQptdXRhdGUoZmwsCiAgICAgICBnYWluID0gZGVwX2RlbGF5IC0gYXJyX2RlbGF5LAogICAgICAgYWlyX2hvdXJzID0gYWlyX3RpbWUgLyA2MCwKICAgICAgIGdhaW5fcGVyX2hvdXIgPSBnYWluIC8gYWlyX2hvdXJzKSAlPiUKICAgIGhlYWQoMikKYGBgCgpgdHJhbnNtdXRlKClgIGtlZXBzIG9ubHkgdGhlIG5ldyB2YXJpYWJsZXMuCgpgYGB7cn0KdHJhbnNtdXRlKGZsaWdodHMsCiAgICAgICAgICBnYWluID0gZGVwX2RlbGF5IC0gYXJyX2RlbGF5LAogICAgICAgICAgYWlyX2hvdXJzID0gYWlyX3RpbWUgLyA2MCwKICAgICAgICAgIGdhaW5fcGVyX2hvdXIgPSBnYWluIC8gYWlyX2hvdXJzKSAlPiUKICAgIGhlYWQoMikKYGBgCgpVc2VmdWwgb3BlcmF0b3JzIGluY2x1ZGU6CgoqIEJhc2ljIGFyaXRobWV0aWMgb3BlcmF0b3JzLCBsb2dhcml0aG1zLCBldGMuCgoqIE1vZHVsYXIgYXJpdGhtZXRpYyBvcGVyYXRvcnMgYCUvJWAgYW5kIGAlJWAKCiogTG9naWNhbCBvcGVyYXRvcnMgYW5kIGNvbXBhcmlzb24gb3BlcmF0b3JzLgoKKiBPZmZzZXQgb3BlcmF0b3JzIGBsZWFkKClgIGFuZCBgbGFnKClgLgoKKiBjdW11bGF0aXZlIGFnZ3JlZ2F0ZXMgc3VjaCBhcyBgY3Vtc3VtKClgLCBgY3VtbWVhbigpYC4KCiogUmFua2luZyBvcGVyYXRvcnMsIHN1Y2ggYXMgYG1pbl9yYW5rKClgLCBgcGVyY2VudF9yYW5rKClgLgoKKiBQcmVkaWNhdGVzIGxpa2UgYGlzLm5hYC4KCkEgcmVhc29uYWJsZSBndWVzcyBpcyB0aGF0IGZsaWdodHMgd2l0aCBgTkFgIHZhbHVlcyBmb3IgYm90aApgZGVwX3RpbWVgIGFuZCBgYXJyX3RpbWVgIHdlcmUgY2FuY2VsbGVkOgoKYGBge3J9CmZsIDwtIG11dGF0ZShmbGlnaHRzLAogICAgICAgICAgICAgY2FuY2VsbGVkID0gaXMubmEoZGVwX3RpbWUpICYgaXMubmEoYXJyX3RpbWUpKQpgYGAKCkZvciB0aGUgYGdleXNlcmAgZGF0YSBzZXQgZnJvbSB0aGUgYE1BU1NgIHBhY2thZ2Ugd2UgdXNlZCBgbGFnYCB0bwpvYnRhaW4gdGhlIGR1cmF0aW9uIG9mIHRoZSBwcmV2aW91cyBlcnVwdGlvbiwKCmBgYHtyfQpkYXRhKGdleXNlciwgcGFja2FnZSA9ICJNQVNTIikKZ2V5c2VyMiA8LSBtdXRhdGUoZ2V5c2VyLAogICAgICAgICAgICAgICAgICBwcmV2X2R1cmF0aW9uID0gbGFnKGR1cmF0aW9uKSkKaGVhZChnZXlzZXIyLCA0KQpgYGAKCmBgYHtyIGdleXNlci1sYWcsIGV2YWwgPSBGQUxTRSwgd2FybmluZyA9IEZBTFNFfQp0aG0gPC0gdGhlbWVfbWluaW1hbCgpICsKICAgIHRoZW1lKHRleHQgPSBlbGVtZW50X3RleHQoc2l6ZSA9IDE2KSkKZ2dwbG90KGdleXNlcjIpICsKICAgIGdlb21fcG9pbnQoYWVzKHggPSBwcmV2X2R1cmF0aW9uLAogICAgICAgICAgICAgICAgICAgeSA9IHdhaXRpbmcpKSArCiAgICB0aG0KYGBgCgpgYGB7ciBnZXlzZXItbGFnLCBlY2hvID0gRkFMU0V9CmBgYAoKVGhlIHRvcCBmaXZlIHZlaGljbGVzIGluIGBjdHlgIGdhcyBtaWxlYWdlICh0aGVyZSBtYXkgYmUgbW9yZSB0aGFuCmZpdmUgaWYgdGhlcmUgaXMgYSB0aWUgZm9yIGZpZnRoIHBsYWNlKToKCmBgYHtyfQptdXRhdGUobmV3bXBnMSwgY3R5X3JhbmsgPSBtaW5fcmFuayhkZXNjKGN0eSkpKSAlPiUKICAgIGZpbHRlcihjdHlfcmFuayA8PSA1KQpgYGAKCkFuIGFsdGVybmF0aXZlIHRvIGV4cGxpY2l0bHkgY29tcHV0aW5nIHJhbmtzIGFuZCBmaWx0ZXJpbmcgaXMgdGhlCmBzbGljZV9tYXgoKWAgdXRpbGl0eSBmdW5jdGlvbjoKCmBgYHtyfQpzbGljZV9tYXgobmV3bXBnMSwgY3R5LCBuID0gNSkKYGBgCgpUaGUgYHBlcmNlbnRfcmFua2AgZnVuY3Rpb24gcHJvZHVjZXMgYSAncGVyY2VudGlsZSByYW5rJyAoYnV0IGJldHdlZW4KMCBhbmQgMSkuCgpUaGUgdG9wIDUlIG9mIGNvdW50cmllcyBpbiBsaWZlIGV4cGVjdGFuY3kgaW4gMjAwNyBmcm9tIHRoZQpgZ2FwbWluZGVyYCBkYXRhIHNldDoKCmBgYHtyfQpsaWJyYXJ5KGdhcG1pbmRlcikKZ20gPC0gZmlsdGVyKGdhcG1pbmRlciwgeWVhciA9PSAyMDA3KQpnbV90b3AgPC0gZmlsdGVyKGdtLCBwZXJjZW50X3JhbmsoZGVzYyhsaWZlRXhwKSkgPD0gMC4wNSkKZ21fdG9wCmBgYAoKQW5kIHRoZSBib3R0b20gNSU6CmBgYHtyfQpmaWx0ZXIoZ20sIHBlcmNlbnRfcmFuayhsaWZlRXhwKSA8PSAwLjA1KQpgYGAKCldpdGhpbiB0aGVzZSByZXN1bHRzIHRoZSBvcmlnaW5hbCByb3cgb3JkZXIgaXMgcmV0YWluZWQuCgpUaGUgYGFycmFuZ2VgIGZ1bmN0aW9uIGNhbiBiZSB1c2VkIHRvIGNoYW5nZSB0aGlzLgoKCiMjIEFycmFuZ2luZyBSb3dzCgpgYXJyYW5nZSgpYCByZW9yZGVycyB0aGUgcm93cyBhY2NvcmRpbmcgdG8gb25lIG9yIG1vcmUgdmFyaWFibGVzLgoKQnkgZGVmYXVsdCB0aGUgb3JkZXIgaXMgc21hbGxlc3QgdG8gbGFyZ2VzdDsgdXNpbmcgYGRlc2MoKWAgcmV2ZXJzZXMgdGhpcy4KCkFkZGl0aW9uYWwgdmFyaWFibGVzIHdpbGwgYmUgdXNlZCB0byBicmVhayB0aWVzLgoKQXJyYW5naW5nIHRoZSB0b3AgNSUgb2YgY291bnRyaWVzIGluIGxpZmUgZXhwZWN0YW5jeSBpbiAyMDA3IGJ5IGxpZmUKZXhwZWN0YW5jeToKCmBgYHtyfQphcnJhbmdlKGdtX3RvcCwgZGVzYyhsaWZlRXhwKSkKYGBgCgpUbyBvcmRlciBieSBjb250aW5lbnQgbmFtZSBhbmQgb3JkZXIgYnkgbGlmZSBleHBlY3RhbmN5IHdpdGhpbiBjb250aW5lbnQ6CgpgYGB7cn0KYXJyYW5nZShnbV90b3AsIGNvbnRpbmVudCwgZGVzYyhsaWZlRXhwKSkKYGBgCgpNaXNzaW5nIHZhbHVlcyBhcmUgc29ydGVkIHRvIHRoZSBlbmQ6CgpgYGB7cn0KZGYgPC0gZGF0YS5mcmFtZSh4ID0gYyg1LCAyLCBOQSwgMykpCmFycmFuZ2UoZGYsIHgpCmFycmFuZ2UoZGYsIGRlc2MoeCkpCmBgYAoKVXNpbmcgYGlzLm5hYCB5b3UgY2FuIGFycmFuZ2UgdG8gcHV0IHRoZW0gYXQgdGhlIGJlZ2lubmluZzoKCmBgYHtyfQphcnJhbmdlKGRmLCBkZXNjKGlzLm5hKHgpKSwgeCkKYGBgCgoKIyMgU3VtbWFyaXppbmcgYW5kIEdyb3VwaW5nCgpgc3VtbWFyaXplKClgIGNvbGxhcHNlcyBhIHRhYmxlIGRvd24gdG8gb25lIHJvdyBvZiBzdW1tYXJpZXMuCgpUaGUgYXZlcmFnZSBhbmQgbWF4aW1hbCBkZXBhcnR1cmUgYW5kIGFycml2YWwgZGVsYXlzIGZvciBmbGlnaHRzLAphbG9uZyB3aXRoIHRoZSB0b3RhbCBudW1iZXIgb2YgZmxpZ2h0czoKCmBgYHtyfQpzdW1tYXJpemUoZmxpZ2h0cywKICAgICAgICAgIGF2ZV9kZXBfZGVsYXkgPSBtZWFuKGRlcF9kZWxheSwgbmEucm0gPSBUUlVFKSwKICAgICAgICAgIGF2ZV9hcnJfZGVsYXkgPSBtZWFuKGFycl9kZWxheSwgbmEucm0gPSBUUlVFKSwKICAgICAgICAgIG1heF9kZXBfZGVsYXkgPSBtYXgoZGVwX2RlbGF5LCBuYS5ybSA9IFRSVUUpLAogICAgICAgICAgbWF4X2Fycl9kZWxheSA9IG1heChhcnJfZGVsYXksIG5hLnJtID0gVFJVRSksCiAgICAgICAgICBuID0gbigpKQpgYGAKCgojIyMgR3JvdXBlZCBTdW1tYXJpZXMKCmBzdW1tYXJpemUoKWAgaXMgbW9zdCBjb21tb25seSB1c2VkIHdpdGggX2dyb3VwZWQgZGF0YV8gdG8gcHJvdmlkZSBncm91cApsZXZlbCBzdW1tYXJpZXMuCgpHcm91cGluZyBkZWxheSBzdW1tYXJpZXMgYnkgZGVzdGluYXRpb246CgpgYGB7ciwgd2FybmluZyA9IEZBTFNFfQpmbF9kZXN0IDwtCiAgICBncm91cF9ieShmbGlnaHRzLCBkZXN0KSAlPiUKICAgIHN1bW1hcml6ZShhdmVfZGVwX2RlbGF5ID0gbWVhbihkZXBfZGVsYXksIG5hLnJtID0gVFJVRSksCiAgICAgICAgICAgICAgYXZlX2Fycl9kZWxheSA9IG1lYW4oYXJyX2RlbGF5LCBuYS5ybSA9IFRSVUUpLAogICAgICAgICAgICAgIG1heF9kZXBfZGVsYXkgPSBtYXgoZGVwX2RlbGF5LCBuYS5ybSA9IFRSVUUpLAogICAgICAgICAgICAgIG1heF9hcnJfZGVsYXkgPSBtYXgoYXJyX2RlbGF5LCBuYS5ybSA9IFRSVUUpLAogICAgICAgICAgICAgIG4gPSBuKCkpICU+JQogICAgdW5ncm91cCgpCmhlYWQoZmxfZGVzdCwgNSkKYGBgCgo8ZGl2IGNsYXNzID0gImFsZXJ0Ij4KQ2FsbGluZyBgdW5ncm91cGAgaXMgbm90IGFsd2F5cyBuZWNlc3NhcnkgYnV0IHNvbWV0aW1lcyBpdCBpcywgc28gaXQKaXMgYSBnb29kIGhhYml0IHRvIGdldCBpbnRvLgo8L2Rpdj4KClRoZSBncm91cCBsZXZlbCBzdW1tYXJpZXMgY2FuIGJlIGZ1cnRoZXIgdHJhbnNmb3JtZWQsIGZvciBleGFtcGxlCnRvIGlkZW50aWZ5IHRoZSB0b3AgMTAgZGVzdGluYXRpb25zIGZvciBhdmVyYWdlIGRlbGF5czoKCmBgYHtyfQptdXRhdGUoZmxfZGVzdCwgcmFuayA9IG1pbl9yYW5rKGRlc2MoYXZlX2RlcF9kZWxheSkpKSAlPiUKICAgIGZpbHRlcihyYW5rIDw9IDEwKQpgYGAKCgojIyMgTWlzc2luZyBWYWx1ZXMKCk1vc3Qgc3VtbWFyaWVzIHdpbGwgYmUgYE5BYCBpZiBhbnkgb2YgdGhlIGRhdGEgdmFsdWVzIGJlaW5nIHN1bW1hcml6ZWQKYXJlIGBOQWAuCgpUeXBpY2FsbHksIHN1bW1hcnkgZnVuY3Rpb25zIGNhbiBiZSBjYWxsZWQgd2l0aCBgbmEucm0gPSBUUlVFYCB0bwpwcm9kdWNlIHRoZSBzdW1tYXJ5IGZvciBhbGwgbm9uLW1pc3NpbmcgdmFsdWVzLgoKYGBge3J9CnN1bW1hcml6ZShmbGlnaHRzLAogICAgICAgICAgYXZlX2RlcF9kZWxheSA9IG1lYW4oZGVwX2RlbGF5LCBuYS5ybSA9IFRSVUUpLAogICAgICAgICAgYXZlX2Fycl9kZWxheSA9IG1lYW4oYXJyX2RlbGF5LCBuYS5ybSA9IFRSVUUpLAogICAgICAgICAgbWF4X2RlcF9kZWxheSA9IG1heChkZXBfZGVsYXksIG5hLnJtID0gVFJVRSksCiAgICAgICAgICBtYXhfYXJyX2RlbGF5ID0gbWF4KGFycl9kZWxheSwgbmEucm0gPSBUUlVFKSwKICAgICAgICAgIG4gPSBuKCkpCmBgYAoKWW91IGNhbiBhbHNvIHVzZSBgZmlsdGVyYCB0byByZW1vdmUgcm93cyB3aXRoIGBOQWAgdmFsdWVzIGluIHRoZQp2YXJpYWJsZXMgdG8gYmUgc3VtbWFyaXplZDoKCmBgYHtyfQpmaWx0ZXIoZmxpZ2h0cywgISBpcy5uYShkZXBfZGVsYXkpLCAhIGlzLm5hKGFycl9kZWxheSkpICU+JQogICAgc3VtbWFyaXplKGF2ZV9kZXBfZGVsYXkgPSBtZWFuKGRlcF9kZWxheSksCiAgICAgICAgICAgICAgYXZlX2Fycl9kZWxheSA9IG1lYW4oYXJyX2RlbGF5KSwKICAgICAgICAgICAgICBtYXhfZGVwX2RlbGF5ID0gbWF4KGRlcF9kZWxheSksCiAgICAgICAgICAgICAgbWF4X2Fycl9kZWxheSA9IG1heChhcnJfZGVsYXkpLAogICAgICAgICAgICAgIG4gPSBuKCkpCmBgYAoKPGRpdiBjbGFzcyA9ICJhbGVydCI+ClRoZSB0d28gYXBwcm9hY2hlcyBkbyBwcm9kdWNlIGRpZmZlcmVudCBjb3VudHMuCjwvZGl2PgoKCiMjIyBDb3VudHMKClRoZSBoZWxwZXIgZnVuY3Rpb24gYG4oKWAgcHJvdmlkZXMgYSBjb3VudCBvZiB0aGUgbnVtYmVyIG9mIHJvd3MgaW4KdGhlIHRhYmxlIGJlaW5nIHN1bW1hcml6ZWQuCgpJbmNsdWRpbmcgY291bnRzIHdpdGggZ3JvdXBlZCBzdW1tYXJpZXMgaXMgdXN1YWxseSBhIGdvb2QgaWRlYS4KClN1bW1hcmllcyBiYXNlZCBvbiBzbWFsbCBjb3VudHMgYXJlIGxpa2VseSB0byBiZSBtdWNoIGxlc3MgcmVsaWFibGUKdGhhbiBvbmVzIGJhc2VkIG9uIGxhcmdlciBjb3VudHMuCgpUaGlzIGlzIHJlZmxlY3RlZCBpbiBoaWdoZXIgdmFyaWFiaWxpdHkgYW1vbmcgYXZlcmFnZXMgZm9yCmRlc3RpbmF0aW9ucyB3aXRoIGZld2VyIGZsaWdodHM6CgpgYGB7ciBhdmVfYXJyX2RlbGF5LCBldmFsID0gRkFMU0V9CmdncGxvdChmbF9kZXN0KSArCiAgICBnZW9tX3BvaW50KGFlcyh4ID0gbiwKICAgICAgICAgICAgICAgICAgIHkgPSBhdmVfYXJyX2RlbGF5KSwKICAgICAgICAgICAgICAgbmEucm0gPSBUUlVFKSArCiAgICB0aG0KYGBgCgpgYGB7ciBhdmVfYXJyX2RlbGF5LCBlY2hvID0gRkFMU0V9CmBgYAoKQSBgcGxvdGx5YCB2ZXJzaW9uIGFsbG93cyB0aGUgaW5kaXZpZHVhbCBkZXN0aW5hdGlvbnMgdG8gYmUgaWRlbnRpZmllZAplYXNpbHk6CgpgYGB7ciBhdmVfYXJyX2RlbGF5X3Bsb3RseSwgZXZhbCA9IEZBTFNFfQpsaWJyYXJ5KHBsb3RseSkKcCA8LSBnZ3Bsb3QoZmxfZGVzdCkgKwogICAgZ2VvbV9wb2ludChhZXMoeCA9IG4sCiAgICAgICAgICAgICAgICAgICB5ID0gYXZlX2Fycl9kZWxheSwKICAgICAgICAgICAgICAgICAgIHRleHQgPSBkZXN0KSwKICAgICAgICAgICAgICAgbmEucm0gPSBGQUxTRSkgKwogICAgdGhtCmdncGxvdGx5KHAsIHRvb2x0aXAgPSAidGV4dCIpCmBgYAoKYGBge3IgYXZlX2Fycl9kZWxheV9wbG90bHksIGVjaG8gPSBGQUxTRSwgd2FybmluZyA9IEZBTFNFfQpgYGAKCkl0IHdvdWxkIGJlIG1vcmUgdXNlci1mcmllbmRseSB0byBzaG93IHRoZSBhaXJwb3J0IG5hbWUgdGhhbiB0aGUgdGhyZWUKbGV0dGVyIEZBQSBjb2RlLgoKCiMjIyBVc2VmdWwgU3VtbWFyaWVzCgpTb21lIHVzZWZ1bCBzdW1tYXJ5IGZ1bmN0aW9uczoKCiogTG9jYXRpb246IGBtZWFuYCwgYG1lZGlhbmAKCiogU3ByZWFkOiBgc2RgLCBgSVFSYCwgYG1hZGAKCiogUmFuazogYG1pbmAsIGBxdWFudGlsZWAsIGBtYXhgCgoqIFBvc2l0aW9uOiBgZmlyc3RgLCBgbnRoYCwgYGxhc3RgCgoqIENvdW50czogYG5gLgoKCiMjIEFpcmxpbmUgQXJyaXZhbCBEZWxheXMKCgojIyMgRGlmZmVyZW50IERlbGF5IE1lYXN1cmVzCgpTb21ldGltZXMgaXQgaXMgdXNlZnVsIHRvIGxvb2sgYXQgc3Vic2V0cyBvciB0cnVuY2F0ZWQgdmVyc2lvbnMgb2YgYQp2YXJpYWJsZS4KCkFpcmxpbmVzIGxpa2UgdG8gYXJyYW5nZSBzY2hlZHVsZXMgc28gdGhleSBhcmUgb2Z0ZW4gZWFybHkuCgpgYGB7ciwgZmlnLmhlaWdodCA9IDQsIGNsYXNzLnNvdXJjZSA9ICJmb2xkLWhpZGUifQpnZ3Bsb3QoZmxpZ2h0cywgYWVzKHggPSBhcnJfZGVsYXksCiAgICAgICAgICAgICAgICAgICAgeSA9IGFmdGVyX3N0YXQoZGVuc2l0eSkpKSArCiAgICBnZW9tX2hpc3RvZ3JhbShiaW53aWR0aCA9IDEsCiAgICAgICAgICAgICAgICAgICBuYS5ybSA9IFRSVUUsCiAgICAgICAgICAgICAgICAgICBmaWxsID0gImRlZXBza3libHVlMyIpICsKICAgIHhsaW0oYygtMTAwLCAyNTApKSArCiAgICB0aG0KYGBgCgpUaGlzIHJlZHVjZXMgdGhlIGF2ZXJhZ2UgYXJyaXZhbCBkZWxheSBidXQgbWF5IG5vdCBoZWxwIHRyYXZlbGVycyBtdWNoLgoKVHdvIG9wdGlvbnM6CgoqIENvbnNpZGVyIG9ubHkgdGhlIHBvc2l0aXZlIGRlbGF5cyBmb3IgdGhlIGF2ZXJhZ2UuCgoqIFRyZWF0IG5lZ2F0aXZlIGRlbGF5cyBhcyB6ZXJvICh6ZXJvLXRydW5jYXRpb24pLgoKVGhlIG1lYW4gb2YgdGhlIHBvc2l0aXZlIGRlbGF5cyBjYW4gYmUgY29tcHV0ZWQgdXNpbmcgX3N1YnNldHRpbmdfOgoKYGBge3IsIGV2YWwgPSBGQUxTRX0KbWVhbihhcnJfZGVsYXlbYXJyX2RlbGF5ID4gMF0pCmBgYAoKVGhlIG1lYW4gb2YgdGhlIHplcm8tdHJ1bmNhdGVkIGRlbGF5cyBjYW4gYmUgY29tcHV0ZWQgdXNpbmcgdGhlIGBwbWF4YApmdW5jdGlvbjoKCmBgYHtyLCBldmFsID0gRkFMU0V9Cm1lYW4ocG1heChhcnJfZGVsYXksIDApKQpgYGAKVGhlIHN1bW1hcmllcyBhcmUKYGBge3J9CmZsX2FyciA8LSBmaWx0ZXIoZmxpZ2h0cywgISBpcy5uYShhcnJfZGVsYXkpKQpmbF9hdnMgPC0KICAgIGdyb3VwX2J5KGZsX2FyciwgZGVzdCkgJT4lCiAgICBzdW1tYXJpemUoYXZlID0gbWVhbihhcnJfZGVsYXkpLAogICAgICAgICAgICAgIGF2ZV9wb3MgPSBtZWFuKGFycl9kZWxheVthcnJfZGVsYXkgPiAwXSksCiAgICAgICAgICAgICAgYXZlX3Bvc196ZXJvID0gbWVhbihwbWF4KGFycl9kZWxheSwgMCkpLAogICAgICAgICAgICAgIG4gPSBuKCkpICU+JQogICAgdW5ncm91cCgpCmhlYWQoZmxfYXZzKQpgYGAKCk9uZSBhcHByb2FjaCB0byB2aXN1YWxpemluZyB0aGUgcmVzdWx0cyB3aXRoIGNvbG9yIGNvZGluZyB0aGUgdGhyZWUKZGlmZmVyZW50IHN1bW1hcmllczoKCiogVmlldyB0aGUgYGZsX2F2c2AgdGFibGUgYXMgYSB3aWRlIGZvcm1hdCB3aXRoIGBhdnNgLCBgYXZlX3Bvc2AsIGFuZAogIGBhdmVfcG9zX3plcm9gIGFzIHRocmVlIGRpZmZlcmVudCBtZWFzdXJlbWVudCB0eXBlcy4KCiogVXNlIGBwaXZvdF9sb25nZXIoKWAgdG8gY29udmVydCB0byBhIGxvbmcgZm9ybWF0IHdpdGggYSB2YXJpYWJsZQogIGB3aGljaGAgdG8gaG9sZCB0aGUgc3VtbWFyeSB0eXBlIGFuZCBhIHZhcmlhYmxlIGBkZWxheWAgdG8KICBob2xkIHRoZSBzdW1tYXJ5IHZhbHVlOgoKYGBge3J9CmxpYnJhcnkodGlkeXIpCmZsIDwtIHBpdm90X2xvbmdlcihmbF9hdnMsIDIgOiA0LAogICAgICAgICAgICAgICAgICAgbmFtZXNfdG8gPSAid2hpY2giLAogICAgICAgICAgICAgICAgICAgdmFsdWVzX3RvID0gImRlbGF5IikKYXJyYW5nZShmbCwgZGVzdCkKYGBgCgpBIHBsb3QgaXMgdGhlbiBlYXN5IHRvIGNyZWF0ZToKCmBgYHtyIGF2ZXNfcGl2b3QsIGV2YWwgPSBGQUxTRX0KZ2dwbG90KGZsLCBhZXMoeCA9IG4sCiAgICAgICAgICAgICAgIHkgPSBkZWxheSwKICAgICAgICAgICAgICAgY29sb3IgPSB3aGljaCkpICsKICAgIGdlb21fcG9pbnQoKSArCiAgICB0aG0KYGBgCgpgYGB7ciBhdmVzX3Bpdm90LCBlY2hvID0gRkFMU0V9CmBgYAoKQW4gYWx0ZXJuYXRpdmU6CgoqIFVzZSBvbmUgbGF5ZXIgcGVyIHN1bW1hcnkgdHlwZS4KCiogU3BlY2lmeSB0aGUgc3VtbWFyeSB0eXBlIGZvciBlYWNoIGxheWVyIGFzIGEgYGNvbG9yYCBhZXN0aGV0aWMgdmFsdWUuCgpgYGB7ciBhdmVzX2xheWVyLCBldmFsID0gRkFMU0V9CmdncGxvdChmbF9hdnMsIGFlcyh4ID0gbikpICsKICAgIGdlb21fcG9pbnQoYWVzKHkgPSBhdmUsCiAgICAgICAgICAgICAgICAgICBjb2xvciA9ICJhdmUiKSkgKwogICAgZ2VvbV9wb2ludChhZXMoeSA9IGF2ZV9wb3MsCiAgICAgICAgICAgICAgICAgICBjb2xvciA9ICJhdmVfcG9zIikpICsKICAgIGdlb21fcG9pbnQoYWVzKHkgPSBhdmVfcG9zX3plcm8sCiAgICAgICAgICAgICAgICAgICBjb2xvciA9ICJhdmVfcG9zX3plcm8iKSkgKwogICAgdGhtCmBgYAoKYGBge3IgYXZlc19sYXllciwgZWNobyA9IEZBTFNFfQpgYGAKClRoZSBvbmUgbWlzc2luZyB2YWx1ZSBjYXNlOgoKYGBge3J9CmluY29tcGxldGVfY2FzZXMoZmxfYXZzKQpgYGAKCgojIyMgRGVsYXkgUHJvcG9ydGlvbnMKClByb3BvcnRpb25zIHNhdGlzZnlpbmcgYSBjb25kaXRpb24gY2FuIGJlIGNvbXB1dGVkIGFzIGEgbWVhbgpvZiB0aGUgbG9naWNhbCBleHByZXNzaW9uIGZvciB0aGUgY29uZGl0aW9uOgoKVGhlIHByb3BvcnRpb24gb2YgYWxsIGZsaWdodHMgdGhhdCB3ZXJlIGRlbGF5ZWQ6CgpgYGB7cn0KbWVhbihmbF9hcnIkYXJyX2RlbGF5ID4gMCkKYGBgCgpHcm91cGVkIGJ5IGRlc3RpbmF0aW9uOgoKYGBge3J9CmZsX3Byb3BzIDwtCiAgICBncm91cF9ieShmbF9hcnIsIGRlc3QpICU+JQogICAgc3VtbWFyaXplKAogICAgICAgIHBkMCA9IG1lYW4oYXJyX2RlbGF5ID4gMCksICAgIyMgcHJvcG9ydGlvbiBkZWxheWVkCiAgICAgICAgcGQxMCA9IG1lYW4oYXJyX2RlbGF5ID4gMTApLCAjIyBtb3JlIHRoYW4gMTAgbWludXRlcwogICAgICAgIHBkMjAgPSBtZWFuKGFycl9kZWxheSA+IDIwKSwgIyMgbW9yZSB0aGFuIDIwIG1pbnV0ZXMKICAgICAgICBuID0gbigpKSAlPiUKICAgIHVuZ3JvdXAoKQpmbF9wcm9wcwpgYGAKClRoZXNlIGNhbiBiZSB2aXN1YWxpemVkIHNpbWlsYXJseToKCmBgYHtyIHByb3BfZGVsYXllZF9waXZvdCwgZXZhbCA9IEZBTFNFfQpwaXZvdF9sb25nZXIoZmxfcHJvcHMsIDIgOiA0LAogICAgICAgICAgICAgbmFtZXNfdG8gPSAid2hpY2giLAogICAgICAgICAgICAgdmFsdWVzX3RvID0gInByb3AiKSAlPiUKICAgIGdncGxvdChhZXMoeCA9IG4sCiAgICAgICAgICAgICAgIHkgPSBwcm9wLAogICAgICAgICAgICAgICBjb2xvciA9IHdoaWNoKSkgKwogICAgZ2VvbV9wb2ludCgpICsKICAgIHRobQpgYGAKCmBgYHtyIHByb3BfZGVsYXllZF9waXZvdCwgZWNobyA9IEZBTFNFfQpgYGAKCgojIyBHcm91cGVkIE11dGF0ZSBhbmQgRmlsdGVyCgpgbXV0YXRlKClgLCBgZmlsdGVyKClgLCBgYXJyYW5nZSgpYCwgYHNsaWNlX21heCgpYCwgYW5kIGBzbGljZV9taW4oKWAKY2FuIGJlIHVzZWQgb24gYSBncm91cCBsZXZlbC4KCgojIyMgU3RhbmRhcmRpemluZyBXaXRoaW4gR3JvdXBzCgpgbXV0YXRlYCB3aXRoaW4gZ3JvdXBzIGNhbiBiZSB1c2VkIGZvciBzdGFuZGFyZGl6aW5nIHdpdGhpbiBhIGdyb3VwLgoKRm9yIGV4YW1wbGUsIHRvIGNvbXB1dGUgdGhlIHByb3BvcnRpb24gb2YgYSBjb250aW5lbnQncyBwb3B1bGF0aW9uCmxpdmluZyBpbiBlYWNoIGNvdW50cnkgZm9yIGVhY2ggeWVhcjoKCmBgYHtyfQpnbSA8LSBncm91cF9ieShnYXBtaW5kZXIsIGNvbnRpbmVudCwgeWVhcikgJT4lCiAgICBtdXRhdGUocG9wID0gYXMubnVtZXJpYyhwb3ApLAogICAgICAgICAgIHBwb3AgPSBwb3AgLyBzdW0ocG9wKSkgJT4lCiAgICB1bmdyb3VwKCkKYGBgCgpGb3IgT2NlYW5pYSAoaW4gdGhpcyBkYXRhIHNldCkgYW5kIHllYXJzIDE5NTIgYW5kIDIwMDc6CgpgYGB7cn0KZ21fb2MgPC0gZmlsdGVyKGdtLCBjb250aW5lbnQgPT0gIk9jZWFuaWEiLCB5ZWFyICVpbiUgYygxOTUyLCAyMDA3KSkgJT4lCiAgICBhcnJhbmdlKHllYXIpCmdtX29jCmBgYAoKQSBxdWljayBjaGVjayB0aGF0IHRoYXQgd2l0aGluIHllYXIgYW5kIGNvbnRpbmVudCB0aGUgcHJvcG9ydGlvbnMgYWRkCnVwIHRvIG9uZToKCmBgYHtyfQpncm91cF9ieShnbV9vYywgY29udGluZW50LCB5ZWFyKSAlPiUgc3VtbWFyaXplKHN1bV9wcG9wID0gc3VtKHBwb3ApKQpgYGAKCkZvY3VzIG9uIHRoZSAyIGNvdW50cmllcyBpbiBlYWNoIGNvbnRpbmVudCB3aXRoIHRoZSBoaWdoZXN0CnByb3BvcnRpb25zIG9mIHRoZSBjb250aW5lbnQncyBwb3B1bGF0aW9uOgoKYGBge3IsIGZpZy53aWR0aCA9IDksIGNsYXNzLnNvdXJjZSA9ICJmb2xkLWhpZGUifQpnbXQyIDwtIGdyb3VwX2J5KGdtLCBjb250aW5lbnQsIHllYXIpICU+JQogICAgc2xpY2VfbWF4KHBwb3AsIG4gPSAyKSAlPiUKICAgIHVuZ3JvdXAoKSAlPiUKICAgIGZpbHRlcihjb250aW5lbnQgIT0gIk9jZWFuaWEiKSAgIyMgZHJvcCBhcyB0aGVyZSBhcmUgb25seSB0d28KZ2dwbG90KGdtdDIsIGFlcyh4ID0geWVhciwgeSA9IHBwb3AsIGdyb3VwID0gY291bnRyeSwgY29sb3IgPSBjb3VudHJ5KSkgKwogICAgZ2VvbV9saW5lKCkgKyBnZW9tX3BvaW50KCkgKwogICAgZmFjZXRfd3JhcCh+IGNvbnRpbmVudCwgc2NhbGVzID0gImZyZWVfeSIpICsKICAgIHRobQpgYGAKCkFub3RoZXIgYXBwcm9hY2g6CgoqIENyZWF0ZSBzZXBhcmF0ZSBwbG90cyBmb3IgZWFjaCBjb250aW5lbnQuCgoqIEFkanVzdCB0aGUgdGhlbWVzIGEgbGl0dGxlLgoKKiBVc2UgYGZhY2V0X3dyYXBgIHRvIGdldCB0aGUgZmFjZXQgbGFiZWwuCgoqIEFycmFuZ2UgdGhlIHBsb3RzLCBlLmcuIHVzaW5nIGBwYXRjaHdvcmtgIG9wZXJhdG9ycy4KCkEgdXNlZnVsIGZlYXR1cmUgaXMgdGhhdCB0aGUgYCUrJWAgb3BlcmF0b3IgY2FuIGJlIHVzZWQgdG8gY2hhbmdlIHRoZQpiYXNlIGRhdGEgZnJhbWUgZm9yIGEgcGxvdDoKCmBgYHtyLCBmaWcuaGVpZ2h0ID0gNi41LCBmaWcud2lkdGggPSA4LCBjbGFzcy5zb3VyY2UgPSAiZm9sZC1oaWRlIn0KcEFmIDwtIGdncGxvdChmaWx0ZXIoZ210MiwgY29udGluZW50ID09ICJBZnJpY2EiKSwKICAgICAgICAgICAgICBhZXMoeCA9IHllYXIsIHkgPSBwcG9wLCBjb2wgPSBjb3VudHJ5KSkgKwogICAgZ2VvbV9saW5lKCkgKyBnZW9tX3BvaW50KCkgKwogICAgZmFjZXRfd3JhcCh+IGNvbnRpbmVudCkgKwogICAgdGhtICsKICAgIHRoZW1lKGxlZ2VuZC5wb3NpdGlvbiA9ICJ0b3AiLCBsZWdlbmQudGl0bGUgPSBlbGVtZW50X2JsYW5rKCkpCnRobV9ueCA8LSB0aGVtZShheGlzLnRpdGxlLnggPSBlbGVtZW50X2JsYW5rKCkpCnRobV9ueSA8LSB0aGVtZShheGlzLnRpdGxlLnkgPSBlbGVtZW50X2JsYW5rKCkpCnBBbSA8LSBwQWYgJSslIGZpbHRlcihnbXQyLCBjb250aW5lbnQgPT0gIkFtZXJpY2FzIikKcEFzIDwtIHBBZiAlKyUgZmlsdGVyKGdtdDIsIGNvbnRpbmVudCA9PSAiQXNpYSIpCnBFdSA8LSBwQWYgJSslIGZpbHRlcihnbXQyLCBjb250aW5lbnQgPT0gIkV1cm9wZSIpICsKICAgIGd1aWRlcyhjb2xvciA9IGd1aWRlX2xlZ2VuZChucm93ID0gMikpCmxpYnJhcnkocGF0Y2h3b3JrKQooKHBBZiArIHRobV9ueCkgfCAocEFtICsgdGhtX254ICsgdGhtX255KSkgLyAocEFzIHwgKHBFdSArIHRobV9ueSkpCmBgYAoKCiMjIyBDaGFuZ2VzIE92ZXIgVGltZQoKQ2hhbmdlcyBvdmVyIHRpbWUgd2l0aGluIGEgY291bnRyeSBjYW4gYWxzbyBiZSBleGFtaW5lZCB3aXRoIGEgZ3JvdXBlZApgbXV0YXRlYCBhbmQgdGhlIGBsYWdgIG9wZXJhdG9yLgoKRm9yIHZhcmlhYmxlcyBjb2xsZWN0ZWQgb3ZlciB0aW1lIHRoZSBgbGFnYCBmdW5jdGlvbiBjYW4gYmUgdXNlZCB0bwpjb21wdXRlIHRoZSBwcmV2aW91cyB2YWx1ZSBvZiBhIHZhcmlhYmxlOgoKYGBge3J9CmdtX3VzIDwtIGZpbHRlcihnYXBtaW5kZXIsCiAgICAgICAgICAgICAgICBjb3VudHJ5ID09ICJVbml0ZWQgU3RhdGVzIikgJT4lCiAgICB0cmFuc211dGUoeWVhciwKICAgICAgICAgICAgICBsaWZlRXhwLAogICAgICAgICAgICAgIHByZXZfbGlmZUV4cCA9IGxhZyhsaWZlRXhwKSkKZ21fdXMKYGBgCgpUaGlzIGNhbiB0aGVuIGJlIHVzZWQgdG8gY29tcHV0ZSB0aGUgY2hhbmdlIGZyb20gb25lIHBlcmlvZCB0byB0aGUgbmV4dDoKCmBgYHtyfQptdXRhdGUoZ21fdXMsCiAgICAgICBsZV9jaGFuZ2UgPSBsaWZlRXhwIC0gcHJldl9saWZlRXhwKQpgYGAKClVzZSB0aGlzIGlkZWEgdG8gY29tcHV0ZSBjaGFuZ2VzIGluIGBsaWZlRXhwYCBhbmQgYGdkcFBlcmNhcGAgZm9yIGFsbApjb3VudHJpZXMgaW4gYSBncm91cGVkIGBtdXRhdGVgOgoKYGBge3J9CmdtIDwtIGdyb3VwX2J5KGdhcG1pbmRlciwgY291bnRyeSkgJT4lCiAgICBtdXRhdGUoCiAgICAgICAgbGVfY2hhbmdlID0gbGlmZUV4cCAtIGxhZyhsaWZlRXhwKSwKICAgICAgICBnZHBfY2hhbmdlID0gZ2RwUGVyY2FwIC0gbGFnKGdkcFBlcmNhcCksCiAgICAgICAgZ2RwX3BjdF9jaGFuZ2UgPQogICAgICAgICAgICAxMDAgKiBnZHBfY2hhbmdlIC8gbGFnKGdkcFBlcmNhcCkpICU+JQogICAgdW5ncm91cCgpCmBgYAoKYHNsaWNlX21heCgpYCBjYW4gdGhlbiBiZSBhcHBsaWVkIHRvIGRhdGEgZ3JvdXBlZCBieSBjb3VudHJ5IHRvCmV4dHJhY3QgdGhlIHJvd3Mgd2l0aCB0aGUgaGlnaGVzdCBgZ2RwX3BjdF9jaGFuZ2VgIGZvciBlYWNoIGNvdW50cnk6CgpgYGB7cn0KZ3JvdXBfYnkoZ20sIGNvdW50cnkpICU+JQogICAgc2xpY2VfbWF4KGdkcF9wY3RfY2hhbmdlLCBuID0gMSkgJT4lCiAgICB1bmdyb3VwKCkgJT4lCiAgICBzZWxlY3QoMSA6IDMsIGNvbnRhaW5zKCJnZHAiKSkKYGBgCgpHcm91cGluZyBieSBjb250aW5lbnQgYWxsb3dzIHRoZSB3b3JzdCBkcm9wIGluIGxpZmUgZXhwZWN0YW5jeSBmb3IKZWFjaCBjb250aW5lbnQgdG8gYmUgZm91bmQ6CgpgYGB7cn0KZ3JvdXBfYnkoZ20sIGNvbnRpbmVudCkgJT4lCiAgICBzbGljZV9taW4obGVfY2hhbmdlLCBuID0gMSkgJT4lCiAgICB1bmdyb3VwKCkgJT4lCiAgICBzZWxlY3QoLWNvbnRhaW5zKCJnZHAiKSkKYGBgCgpXaXRob3V0IGdyb3VwaW5nLCB0aGUgcm93IHdpdGggYmVzdCBwZXJjZW50IEdEUCBjaGFuZ2Ugb3ZlcmFsbCBjYW4gYmUgY29tcHV0ZWQ6CgpgYGB7cn0Kc2xpY2VfbWF4KGdtLCBnZHBfcGN0X2NoYW5nZSwgbiA9IDEpICU+JQogICAgc2VsZWN0KGNvdW50cnksIGNvbnRpbmVudCwgeWVhciwgZ2RwX3BjdF9jaGFuZ2UpCmBgYAoKVGhlIGNvbXB1dGF0aW9uIG9mIHRoZSBiZXN0IEdEUCBncm93dGggeWVhcnMgYnkgY291bnRyeSBjYW4gYmUKY29tYmluZWQgaW50byBhIHNpbmdsZSBwaXBlOgoKYGBge3IsIHJlc3VsdHMgPSAiaGlkZSJ9Cmdyb3VwX2J5KGdhcG1pbmRlciwgY291bnRyeSkgJT4lCiAgICBtdXRhdGUoZ2RwX2NoYW5nZSA9IGdkcFBlcmNhcCAtIGxhZyhnZHBQZXJjYXApLAogICAgICAgICAgIGdkcF9wY3RfY2hhbmdlID0gMTAwICogZ2RwX2NoYW5nZSAvIGxhZyhnZHBQZXJjYXApKSAlPiUKICAgIHVuZ3JvdXAoKSAlPiUKICAgIGdyb3VwX2J5KGNvdW50cnkpICU+JQogICAgc2xpY2VfbWF4KGdkcF9wY3RfY2hhbmdlLCBuID0gMSkgJT4lCiAgICB1bmdyb3VwKCkgJT4lCiAgICBzZWxlY3QoMSA6IDMsIGNvbnRhaW5zKCJnZHAiKSkKYGBgCgpTaW5jZSB0aGUgZ3JvdXBpbmdzIG9mIHRoZSB0d28gc3RlcHMgYXJlIHRoZSBzYW1lIGFuZCBncm91cGVkIGBtdXRhdGVgCnByZXNlcnZlcyB0aGUgZ3JvdXAgc3RydWN0dXJlLCB0aGUgbWlkZGxlIGB1bmdyb3VwYC9gZ3JvdXBfYnlgIGNhbiBiZQpkcm9wcGVkOgoKYGBge3IsIHJlc3VsdHMgPSAiaGlkZSJ9Cmdyb3VwX2J5KGdhcG1pbmRlciwgY291bnRyeSkgJT4lCiAgICBtdXRhdGUoZ2RwX2NoYW5nZSA9IGdkcFBlcmNhcCAtIGxhZyhnZHBQZXJjYXApLAogICAgICAgICAgIGdkcF9wY3RfY2hhbmdlID0gMTAwICogZ2RwX2NoYW5nZSAvIGxhZyhnZHBQZXJjYXApKSAlPiUKICAgIHNsaWNlX21heChnZHBfcGN0X2NoYW5nZSwgbiA9IDEpICU+JQogICAgdW5ncm91cCgpICU+JQogICAgc2VsZWN0KDEgOiAzLCBjb250YWlucygiZ2RwIikpCmBgYAoKPGRpdiBjbGFzcyA9ICJhbGVydCI+CllvdSBuZWVkIHRvIGJlIHZlcnkgY2FyZWZ1bCBtYWtpbmcgYW4gb3B0aW1pemF0aW9uIGxpa2UgdGhpcyEKPC9kaXY+CgoKIyMgSm9pbnMKCjwhLS0gQ2hlY2sgb3V0IGh0dHBzOi8vd3d3LmdhcnJpY2thZGVuYnVpZS5jb20vcHJvamVjdC90aWR5ZXhwbGFpbi8gLS0+CgpUaGlzIGludGVyYWN0aXZlIHBsb3QgdXNlcyB0aGUgYHRvb2x0aXBgIHRvIHNob3cgdGhlIGFpcnBvcnQgY29kZSBmb3IKdGhlIGRlc3RpbmF0aW9uOgoKYGBge3IsIGNsYXNzLnNvdXJjZSA9ICJmb2xkLWhpZGUifQpsaWJyYXJ5KHBsb3RseSkKcCA8LSBnZ3Bsb3QoZmxfZGVzdCkgKwogICAgZ2VvbV9wb2ludChhZXMoeCA9IG4sCiAgICAgICAgICAgICAgICAgICB5ID0gYXZlX2Fycl9kZWxheSwKICAgICAgICAgICAgICAgICAgIHRleHQgPSBkZXN0KSkgKwogICAgdGhtCmdncGxvdGx5KHAsIHRvb2x0aXAgPSAidGV4dCIpCmBgYAoKQSBuaWNlciBhcHByb2FjaCB3b3VsZCBiZSB0byBzaG93IHRoZSBhaXJwb3J0IG5hbWUuCgpUaGUgYGFpcnBvcnRzYCB0YWJsZSBwcm92aWRlcyB0aGlzIGluZm9ybWF0aW9uOgoKYGBge3J9CmhlYWQoYWlycG9ydHMsIDIpCmBgYAoKV2UgbmVlZCB0byBnZXQgdGhpcyBpbmZvcm1hdGlvbiBpbnRvIHRoZSBgZmxfZGVzdGAgZGF0YSBmcmFtZS4KCkFzIGFub3RoZXIgZXhhbXBsZSwgd2UgbWlnaHQgd2FudCB0byBleHBsb3JlIHdoZXRoZXIgYWlyY3JhZnQgYWdlCmlzIHJlbGF0ZWQgdG8gZGVsYXlzLgoKVGhlIGBwbGFuZXNgIHRhYmxlIHByb3ZpZGVzIHRoZSB5ZWFyIG9mIG1hbnVmYWN0dXJlOgoKYGBge3J9CmhlYWQocGxhbmVzLCAyKQpgYGAKCkFnYWluLCB3ZSBuZWVkIHRvIG1lcmdlIHRoaXMgaW5mb3JtYXRpb24gd2l0aCB0aGUgZGF0YSBpbiB0aGUgYGZsaWdodHNgIHRhYmxlLgoKT25lIHdheSB0byBkbyB0aGlzIGlzIHdpdGggYSBfam9pbiBvcGVyYXRpb25fLgoKSm9pbnMgYXJlIGEgY29tbW9uIG9wZXJhdGlvbiBpbiBkYXRhIGJhc2VzLgoKKiBNdXJlbGwncyBfRGF0YSBUZWNobm9sb2dpZXNfIGJvb2sgZGVzY3JpYmVzIGRhdGEgYmFzZXMgaW4KICBTZWN0aW9uIDUuNiBhbmQgZGF0YSBiYXNlIHF1ZXJpZXMgdXNpbmcgU1FMIGluIENoYXB0ZXIgNy4KCiogX1IgZm9yIERhdGEgU2NpZW5jZV8gZGVzY3JpYmVzIHJlbGF0aW9uYWwgZGF0YSBpbgogIFtDaGFwdGVyIDEzXShodHRwczovL3I0ZHMuaGFkLmNvLm56L3JlbGF0aW9uYWwtZGF0YS5odG1sKS4KCmBkcGx5cmAgZGlzdGluZ3Vpc2hlcyBiZXR3ZWVuIHR3byBraW5kcyBvZiBqb2luczoKCiogX211dGF0aW5nIGpvaW5zXzsKCiogX2ZpbHRlcmluZyBqb2luc18uCgpUaGVzZSBbYW5pbWF0ZWQKZXhwbGFuYXRpb25zXShodHRwczovL3d3dy5nYXJyaWNrYWRlbmJ1aWUuY29tL3Byb2plY3QvdGlkeWV4cGxhaW4vKSBtYXkKYmUgaGVscGZ1bC4KCgojIyMgTXV0YXRpbmcgSm9pbnMKCk11dGF0aW5nIGpvaW5zIGNvbWJpbmUgdmFyaWFibGVzIGFuZCByb3dzIGZyb20gdHdvIHRhYmxlcyBieSBtYXRjaGluZwpyb3dzIG9uIF9rZXlzXy4KCiogQSBfcHJpbWFyeSBrZXlfIGlzIGEgdmFyaWFibGUsIG9yIGNvbWJpbmF0aW9uIG9mIHZhcmlhYmxlcywgaW4gYQogIHRhYmxlIHRoYXQgdW5pcXVlbHkgaWRlbnRpZnkgdGhlIHJvd3Mgb2YgdGhlIHRhYmxlLgoKKiBBIF9mb3JlaWduIGtleV8gaXMgYSB2YXJpYWJsZSwgb3IgY29tYmluYXRpb24gb2YgdmFyaWFibGVzLCB0aGF0CiAgdW5pcXVlbHkgaWRlbnRpZnkgYSByb3cgaW4gc29tZSB0YWJsZSAodXN1YWxseSwgYnV0IG5vdCBuZWNlc3NhcmlseSwgYQogIGRpZmZlcmVudCB0YWJsZSkuCgpUaGUgc2ltcGxlc3Qgam9pbiBvcGVyYXRpb24gaXMgYW4gX2lubmVyIGpvaW5fOiBPbmx5IHJvd3MgZm9yIHdoaWNoCnRoZSBrZXkgYXBwZWFycyBpbiBib3RoIHRhYmxlcyBhcmUgcmV0YWluZWQuCgpgYGB7ciwgZWNobyA9IEZBTFNFLCBvdXQud2lkdGggPSAiNzAlIn0Ka25pdHI6OmluY2x1ZGVfZ3JhcGhpY3MoSU1HKCJqb2luLWlubmVyLnBuZyIpKQpgYGAKCl9PdXRlciBqb2luc18gcmV0YWluIHNvbWUgb3RoZXIgcm93cyB3aXRoIGBOQWAgdmFsdWVzIGZvciBuZXcgdmFyaWFibGVzOgoKYGBge3IsIGVjaG8gPSBGQUxTRSwgb3V0LndpZHRoID0gIjcwJSJ9CmtuaXRyOjppbmNsdWRlX2dyYXBoaWNzKElNRygiam9pbi1vdXRlci5wbmciKSkKYGBgCgpJbm5lciBhbmQgbGVmdCBqb2lucyBhcmUgdGhlIG1vc3QgY29tbW9uLgoKVGhlIGtleSBvZiBvbmUgb2YgdGhlIHRhYmxlcyBzaG91bGQgdXN1YWxseSBiZSBhIF9wcmltYXJ5IGtleV8gdGhhdAp1bmlxdWVseSBpZGVudGlmaWVzIHRoZSB0YWJsZSdzIHJvd3MuCgpGb3IgYSBsZWZ0IGpvaW4sIHRoZSByaWdodCB0YWJsZSBrZXkgc2hvdWxkIHVzdWFsbHkgdXNlIGEgcHJpbWFyeSBrZXkuCgpJdCBpcyB1c3VhbGx5IGEgZ29vZCBpZGVhIHRvIG1ha2Ugc3VyZSBhIHByaW1hcnkga2V5IGNhbmRpZGF0ZSByZWFsbHkKZG9lcyB1bmlxdWVseSBpZGVudGlmeSByb3dzLgoKRm9yIHRoZSBgcGxhbmVzYCB0YWJsZSB0aGUgYHRhaWxudW1gIHZhcmlhYmxlIGlzIGEgcHJpbWFyeSBrZXk7CgpgYGB7cn0KY291bnQocGxhbmVzLCB0YWlsbnVtKSAlPiUgZmlsdGVyKG4gPiAxKQpgYGAKCkZvciB0aGUgYGZsaWdodHNgIGRhdGEgYSBwcmltYXJ5IGtleSBuZWVkcyB0byB1c2UgbXVsdGlwbGUgdmFyaWFibGVzLgoKQSByZWFzb25hYmxlIGd1ZXNzIGlzIHRvIHVzZSB0aGUgZGF0ZSwgY2FycmllciBhbmQgZmxpZ2h0IG51bWJlciwgYnV0CnRoaXMgaXNuJ3QgcXVpdGUgZW5vdWdoOgoKYGBge3J9CmNvdW50KGZsaWdodHMsIHllYXIsIG1vbnRoLCBkYXksIGZsaWdodCwgY2FycmllcikgJT4lCiAgICBmaWx0ZXIobiA+IDEpICU+JQogICAgaGVhZCgyKQpgYGAKCkFkZGluZyB0aGUgYG9yaWdpbmAgYWlycG9ydCBjb2RlIGRvZXMgcHJvZHVjZSBhIHByaW1hcnkga2V5OgoKYGBge3J9CmNvdW50KGZsaWdodHMsIHllYXIsIG1vbnRoLCBkYXksIGZsaWdodCwgY2Fycmllciwgb3JpZ2luKSAlPiUKICAgIGZpbHRlcihuID4gMSkKYGBgCgpUbyByZWxhdGUgZGVsYXlzIHRvIGFpcmNyYWZ0IGFnZSB3ZSBjYW4gc3RhcnQgd2l0aCBzdW1tYXJpZXMgYnkKYHRhaWxudW1gIGFuZCBgeWVhcmAgKGB5ZWFyYCB3b3VsZCBiZSBuZWVkZWQgaWYgd2UgaGFkIGRhdGEgZm9yIG1vcmUKdGhhbiAyMDEzKToKCmBgYHtyfQpmbCA8LSBmaWx0ZXIoZmxpZ2h0cywgISBpcy5uYShkZXBfdGltZSksICEgaXMubmEoYXJyX3RpbWUpKSAlPiUKICAgIGdyb3VwX2J5KHllYXIsIHRhaWxudW0pICU+JQogICAgc3VtbWFyaXplKGF2ZV9kZXBfZGVsYXkgPSBtZWFuKGRlcF9kZWxheSwgbmEucm0gPSBUUlVFKSwKICAgICAgICAgICAgICBhdmVfYXJyX2RlbGF5ID0gbWVhbihhcnJfZGVsYXksIG5hLnJtID0gVFJVRSksCiAgICAgICAgICAgICAgbiA9IG4oKSkgJT4lCiAgICB1bmdyb3VwKCkKYGBgCgpgYGB7cn0KaGVhZChmbCkKYGBgCgpBIHJlZHVjZWQgYHBsYW5lc2AgdGFibGUgd2l0aCBgeWVhcmAgcmVuYW1lZCB0byBhdm9pZCBhIGNvbmZsaWN0OgoKYGBge3J9CnBsIDwtIHNlbGVjdChwbGFuZXMsIHRhaWxudW0sIHBsYW5lX3llYXIgPSB5ZWFyKQpgYGAKCmBgYHtyfQpwbApgYGAKCk5vdwoKKiB1c2UgYSBsZWZ0IGpvaW4gd2l0aCBgdGFpbG51bWAgYXMgdGhlIGtleSB0byBicmluZyB0aGUgYHBsYW5lX3llYXJgCiAgdmFsdWUgaW50byB0aGUgc3VtbWFyeSB0YWJsZQoKKiBhbmQgdGhlbiBjb21wdXRlIHRoZSBwbGFuZSBhZ2U6CgpgYGB7cn0KZmxfYWdlIDwtIGxlZnRfam9pbihmbCwgcGwsICJ0YWlsbnVtIikgJT4lCiAgICBtdXRhdGUocGxhbmVfYWdlID0geWVhciAtIHBsYW5lX3llYXIpCmZsX2FnZQpgYGAKYGBge3IsIGluY2x1ZGUgPSBGQUxTRX0KIyMgbm90IHN1cmUgaWYgdGhpcyBpcyBzdGlsbCBuZWVkZWQKZmwgPC0gZmxfYWdlCmBgYAoKQW4gaW5pdGlhbCBwbG90OgoKYGBge3IgcGxhbmUtYWdlLWluaXRpYWwsIGV2YWwgPSBGQUxTRX0KZ2dwbG90KGZsX2FnZSwKICAgICAgIGFlcyh4ID0gcGxhbmVfYWdlLAogICAgICAgICAgIHkgPSBhdmVfZGVwX2RlbGF5KSkgKwogICAgZ2VvbV9wb2ludCgpICsKICAgIHRobQpgYGAKCmBgYHtyIHBsYW5lLWFnZS1pbml0aWFsLCBlY2hvID0gRkFMU0UsIHdhcm5pbmcgPSBGQUxTRX0KYGBgCgpKaXR0ZXJpbmcgYW5kIGFkanVzdGluZyB0aGUgcG9pbnQgc2l6ZSBtYXkgaGVscDoKCmBgYHtyIHBsYW5lLWFnZS1qaXR0ZXIsIGV2YWwgPSBGQUxTRX0KZ2dwbG90KGZsX2FnZSwKICAgICAgIGFlcyh4ID0gcGxhbmVfYWdlLAogICAgICAgICAgIHkgPSBhdmVfZGVwX2RlbGF5KSkgKwogICAgZ2VvbV9wb2ludChwb3NpdGlvbiA9ICJqaXR0ZXIiLAogICAgICAgICAgICAgICBzaXplID0gMC4xKSArCiAgICB0aG0KYGBgCgpgYGB7ciBwbGFuZS1hZ2Utaml0dGVyLCBlY2hvID0gRkFMU0UsIHdhcm5pbmcgPSBGQUxTRX0KYGBgCgpBZGRpbmcgYSBzbW9vdGggYW5kIGFkanVzdGluZyB0aGUgcmFuZ2VzIG1heSBhbHNvIGhlbHA6CgpgYGB7ciwgcGxhbmUtYWdlLXNtb290aCwgZXZhbCA9IEZBTFNFfQpnZ3Bsb3QoZmxfYWdlLAogICAgICAgYWVzKHggPSBwbGFuZV9hZ2UsCiAgICAgICAgICAgeSA9IGF2ZV9kZXBfZGVsYXkpKSArCiAgICBnZW9tX3BvaW50KHBvc2l0aW9uID0gImppdHRlciIsCiAgICAgICAgICAgICAgIHNpemUgPSAwLjEpICsKICAgIGdlb21fc21vb3RoKCkgKwogICAgeGxpbSgwLCAzMCkgKwogICAgeWxpbShjKC0yMCwgODApKSArCiAgICB0aG0KYGBgCgpgYGB7ciwgcGxhbmUtYWdlLXNtb290aCwgZWNobyA9IEZBTFNFLCB3YXJuaW5nID0gRkFMU0V9CmBgYAoKQW5vdGhlciBvcHRpb24gaXMgdG8gY29tcHV0ZSBtZWFucyBvciBtZWRpYW5zIHdpdGhpbiB5ZWFyczoKCmBgYHtyIHBsYW5lLWFnZS1tZWFucywgZXZhbCA9IEZBTFNFfQpmbG0gPC0gZ3JvdXBfYnkoZmxfYWdlLCBwbGFuZV9hZ2UpICU+JQogICAgc3VtbWFyaXplKGF2ZV9kZXBfZGVsYXkgPQogICAgICAgICAgICAgICAgICBtZWFuKGF2ZV9kZXBfZGVsYXksCiAgICAgICAgICAgICAgICAgICAgICAgbmEucm0gPSBUUlVFKSkgJT4lCiAgICB1bmdyb3VwKCkKZ2dwbG90KGZsbSwKICAgICAgIGFlcyh4ID0gcGxhbmVfYWdlLAogICAgICAgICAgIHkgPSBhdmVfZGVwX2RlbGF5KSkgKwogICAgZ2VvbV9wb2ludChuYS5ybSA9IFRSVUUpICsKICAgIHRobQpgYGAKCmBgYHtyIHBsYW5lLWFnZS1tZWFucywgZWNobyA9IEZBTFNFfQpgYGAKCkJvdGggdGhlIHNtb290aCBhbmQgdGhlIG1lYW5zIHNob3cgYSBtb2Rlc3QgaW5jcmVhc2Ugb3ZlciB0aGUgZmlyc3QgMTAgeWVhcnMuCgpBIGxlZnQgam9pbiBjYW4gYWxzbyBiZSB1c2VkIHRvIGFkZCB0aGUgYWlycG9ydCBuYW1lIGZyb20gdGhlIGFpcnBvcnQKdGFibGUgdG8gdGhlIGBmbF9kZXN0YCB0YWJsZS4KClRoZSBwcmltYXJ5IGtleSBpbiB0aGUgYGFpcnBvcnRgIHRhYmxlIGlzIHRoZSB0aHJlZS1sZXR0ZXIgYWlycG9ydCBjb2RlCmluIHRoZSBgZmFhYCB2YXJpYWJsZToKCmBgYHtyfQpoZWFkKGFpcnBvcnRzLCAyKQpgYGAKClRoZSBmb3JlaWduIGtleSBpbiB0aGUgYGZsX2Rlc3RgIHRhYmxlIHRvIG1hdGNoIGlzIHRoZSBgZGVzdGAgdmFyaWFibGU6CgpgYGB7cn0KaGVhZChmbF9kZXN0LCAyKQpgYGAKClRoZSBgbGVmdF9qb2luYCBmdW5jdGlvbiBhbGxvd3MgdGhpcyBsaW5rIHRvIGJlIG1hZGUgYnkgc3BlY2lmeWluZyB0aGUKa2V5IGFzIGBjKCJkZXN0IiA9ICJmYWEiKWA6CgpgYGB7cn0KZmxfZGVzdCA8LSBsZWZ0X2pvaW4oZmxfZGVzdCwKICAgICAgICAgICAgICAgICAgICAgc2VsZWN0KGFpcnBvcnRzLCBmYWEsIGRlc3RfbmFtZSA9IG5hbWUpLAogICAgICAgICAgICAgICAgICAgICBjKCJkZXN0IiA9ICJmYWEiKSkKc2VsZWN0KGZsX2Rlc3QsIGRlc3RfbmFtZSwgZXZlcnl0aGluZygpKSAlPiUgaGVhZCgyKQpgYGAKCk5vdyBhIHRvb2x0aXAgY2FuIHByb3ZpZGUgYSBtb3JlIGFjY2Vzc2libGUgbGFiZWw6CgpgYGB7ciBhdmVfYXJyX2Rlc3RfbmFtZSwgZXZhbCA9IEZBTFNFfQpwIDwtIGdncGxvdChmbF9kZXN0KSArCiAgICBnZW9tX3BvaW50KGFlcyh4ID0gbiwKICAgICAgICAgICAgICAgICAgIHkgPSBhdmVfYXJyX2RlbGF5LAogICAgICAgICAgICAgICAgICAgdGV4dCA9IHBhc3RlKGRlc3QsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZGVzdF9uYW1lLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHNlcCA9ICI6ICIpKSkgKwogICAgdGhtCmdncGxvdGx5KHAsIHRvb2x0aXAgPSAidGV4dCIpCmBgYAoKYGBge3IgYXZlX2Fycl9kZXN0X25hbWUsIGVjaG8gPSBGQUxTRSwgd2FybmluZyA9IEZBTFNFfQpgYGAKCgojIyMgRmlsdGVyaW5nIEpvaW5zCgpGaWx0ZXJpbmcgam9pbnMgb25seSBhZmZlY3Qgcm93czsgdGhleSBkbyBub3QgYWRkIHZhcmlhYmxlcy4KCiogYHNlbWlfam9pbih4LCB5KWAgcmV0dXJucyBhbGwgcm93cyBpbiBgeGAgd2l0aCBhIG1hdGNoIGluIGB5YDoKCmBgYHtyLCBlY2hvID0gRkFMU0UsIG91dC53aWR0aCA9ICI3MCUifQprbml0cjo6aW5jbHVkZV9ncmFwaGljcyhJTUcoImpvaW4tc2VtaS5wbmciKSkKYGBgCgoqIGBhbnRpX2pvaW4oeCwgeSlgIHJldHVybnMgYWxsIHJvd3MgaW4gYHhgIHdpdGhvdXQgYSBtYXRjaCBpbiBgeWA6CgpgYGB7ciwgZWNobyA9IEZBTFNFLCBvdXQud2lkdGggPSAiNzAlIn0Ka25pdHI6OmluY2x1ZGVfZ3JhcGhpY3MoSU1HKCJqb2luLWFudGkucG5nIikpCmBgYAoKYHNlbWlfam9pbmAgaXMgdXNlZnVsIGZvciBtYXRjaGluZyBmaWx0ZXJlZCByZXN1bHRzIGJhY2sgdG8gYSB0YWJsZS4KClRoZSBwbG90IG9mIGRlbGF5IGFnYWluc3QgYWdlIHNob3dlZCBzb21lIHN1cnByaXNpbmdseSBvbGQgYWlyY3JhZnQ6CgpgYGB7cn0KdG9wX2FnZSA8LSBzbGljZV9tYXgoZmxfYWdlLCBwbGFuZV9hZ2UsIG4gPSAyKQp0b3BfYWdlCmBgYAoKV2UgY2FuIGlkZW50aWZ5IHRoZXNlIGFpcmNyYWZ0IHdpdGggYSBzZW1pLWpvaW46CgpgYGB7cn0Kc2VtaV9qb2luKHBsYW5lcywgdG9wX2FnZSwgInRhaWxudW0iKQpgYGAKClRoZSBjb3JyZXNwb25kaW5nIGZsaWdodHMgY2FuIGFsc28gYmUgcGlja2VkIG91dCB3aXRoIGEgc2VtaS1qb2luOgoKYGBge3J9CnRvcF9hZ2VfZmxpZ2h0cyA8LSBzZW1pX2pvaW4oZmxpZ2h0cywgdG9wX2FnZSwgInRhaWxudW0iKQpoZWFkKHRvcF9hZ2VfZmxpZ2h0cywgMikKYGBgCgpMb29raW5nIGF0IHRoZSBjYXJyaWVycyBvciBkZXN0aW5hdGlvbnMgbWlnaHQgaGVscDoKCmBgYHtyfQpzZWxlY3QodG9wX2FnZV9mbGlnaHRzLCBjYXJyaWVyLCBkZXN0LCBldmVyeXRoaW5nKCkpICU+JSBoZWFkKDQpCmBgYAoKVGhlIGNhcnJpZXIgc3VtbWFyeToKCmBgYHtyfQpjb3VudCh0b3BfYWdlX2ZsaWdodHMsIGNhcnJpZXIpCmBgYAoKQSBjaGVjayBvZiB0aGUgZG9jdW1lbnRhdGlvbiBmb3IgYHBsYW5lc2Agc2hvd3MgdGhhdCBBbWVyaWNhbiBhbmQgb25lCm90aGVyIGFpcmxpbmUgcmVwb3J0IF9mbGVldCBudW1iZXJzXyByYXRoZXIgdGhhbiB0YWlsIG51bWJlcnMuCgpBbnRpLWpvaW5zIGFyZSB1c2VmdWwgZm9yIGRldGVjdGluZyBqb2luIHByb2JsZW1zLgoKRm9yIGV4YW1wbGUsIG1hbnkgZmxpZ2h0cyBoYXZlIHRhaWwgbnVtYmVycyB0aGF0IGRvIG5vdCBtYXRjaCBlbnRyaWVzCmluIGBwbGFuZXNgOgoKYGBge3J9CmZsIDwtIGFudGlfam9pbihmbGlnaHRzLCBwbGFuZXMsICJ0YWlsbnVtIikKbnJvdyhmbCkKYGBgCgpgYGB7cn0KY291bnQoZmwsIHRhaWxudW0pCmBgYAoKCiMjIyBKb2luIElzc3VlcwoKKiBNYWtlIHN1cmUgYSBwcmltYXJ5IGtleSB5b3Ugd2FudCB0byB1c2UgaGFzIG5vIG1pc3NpbmcgdmFsdWVzLgoKKiBNYWtlIHN1cmUgYSBwcmltYXJ5IGtleSB5b3Ugd2FudCB0byB1c2UgcmVhbGx5IHVuaXF1ZWx5IGlkZW50aWZpZXMKICBhIHJvdy4KCiogTWFrZSBzdXJlIGV2ZXJ5IHZhbHVlIG9mIHlvdXIgZm9yZWlnbiBrZXkgeW91IHdhbnQgdG8gdXNlIHRvIGxpbmsgdG8KICBhbm90aGVyIHRhYmxlIG1hdGNoZXMgYSByb3cgaW4gdGhlIGxpbmtlZCB0YWJsZSAoYGFudGlfam9pbmAgaXMgYQogIGdvb2Qgd2F5IHRvIGRvIHRoaXMpLgoKKiBDaGVjayB0aGF0IHlvdXIgZm9yZWlnbiBrZXkgaGFzIG5vIG1pc3NpbmcgdmFsdWVzLgoKPCEtLQotIG1vcmUgZXhhbXBsZXMKICAgIC0gbWF5YmUgc2VwYXJhdGUgc2VjdGlvbgogICAgLSB3aXRoIHBsb3RzCi0gam9pbnMKICAgIC0gbW92ZSBvdmVyIHNvbWUgb2YgdGhlIGxhdGVyIHN0dWZmCiAgICAtIG9yIG1heWJlIHNlcGFyYXRlIGRvY3VtZW50Ci0tPgoKCiMjIERhdGEgQmFzZSBJbnRlcmZhY2UKCkN1cnJlbnQgZGF0YSBvbiBhaXJsaW5lIGZsaWdodHMgaW4gdGhlIFVTIGlzIGF2YWlsYWJsZSBmcm9tIHRoZSBCdXJlYXUKb2YgVHJhbnNwb3J0YXRpb24gU3RhdGlzdGljcyBhdAoKPGh0dHA6Ly93d3cudHJhbnN0YXRzLmJ0cy5nb3YvRExfU2VsZWN0RmllbGRzLmFzcD9UYWJsZV9JRD0yMzY+CjwhLS1odHRwczovL3RyYW5zdGF0cy5idHMuZ292L1BSRVpJUC9fT25fVGltZV9QZXJmb3JtYW5jZV8yMDE3XzEuemlwIC0tPgoKVGhlIGRhdGEgYXJlIGF2YWlsYWJsZSBpbiBvbmUgY29tcHJlc3NlZCBDU1YgZmlsZSBwZXIgbW9udGguCgpMb2FkZWQgaW50byBSLCBvbmUgbW9udGggb2YgZGF0YSByZXF1aXJlcyBhYm91dCAzMDAgTWIgb2YgbWVtb3J5LgoKRGF0YSBmb3IgMTk4Ny0tMjAwOCB3YXMgY29tcGlsZWQgZm9yIGEgRGF0YUV4cG8gY29tcGV0aXRpb24gb2YgdGhlCl9BbWVyaWNhbiBTdGF0aXN0aWNhbCBBc3NvY2lhdGlvbl8gaGVsZCBpbiAyMDA5LgoKVGhlIGRhdGEgYXJlIGF2YWlsYWJsZSBvbiB0aGUgTGludXggc3lzdGVtcyBpbiB0aGUgZGlyZWN0b3J5CmAvZ3JvdXAvc3RhdHNvZnQvZGF0YS9EYXRhRXhwbzA5YCBhcyBjb21wcmVzc2VkIENTViBmaWxlcyBhbmQgYXMgYQpTUUxpdGUgZGF0YSBiYXNlLgoKTG9hZGluZyB0aGUgZGF0YSBpbnRvIFIgd291bGQgcmVxdWlyZSBhYm91dCA3MCBHYiBvZiBtZW1vcnksIHdoaWNoIGlzCm5vdCBwcmFjdGljYWwuCgpVc2luZyB0aGUgZGF0YSBiYXNlIGlzIGEgZ29vZCBhbHRlcm5hdGl2ZS4KClNRTGl0ZSBpcyBhIHNpbXBsZSByZWxhdGlvbmFsIGRhdGEgYmFzZSB0aGF0IHdvcmtzIHdpdGggZGF0YSBzdG9yZWQgb24KYSBsb2NhbCBkaXNrLgoKTW9yZSBzb3BoaXN0aWNhdGVkIGRhdGEgYmFzZXMgYWxsb3cgZGF0YSB0byBiZSBzdG9yZWQgb24gbXVsdGlwbGUKbWFjaGluZXMgb3IgaW4gdGhlIGNsb3VkCgpUaGVzZSBkYXRhIGJhc2VzIGFyZSB1c3VhbGx5IGFjY2Vzc2VkIGJ5IGNvbm5lY3RpbmcgdG8gYSBzZXJ2ZXIsCndoaWNoIHVzdWFsbHkgaW52b2x2ZXMgYW4gYXV0aGVudGljYXRpb24gcHJvY2Vzcy4KCkZvciBTUUxpdGUgdGhlIGNvbm5lY3Rpb24gaXMgc2V0IHVwIGJ5IGxpbmtpbmcgdG8gYSBmaWxlLgoKVGhlIFJTUUxpdGUgcGFja2FnZSBwcm92aWRlcyBhbiBpbnRlcmZhY2UgdG8gU1FMaXRlLgoKVGhlIGBEQklgIHBhY2thZ2UgcHJvdmlkZXMgYSBoaWdoIGxldmVsIGludGVyZmFjZSBmb3IgdXNpbmcgZGF0YSBiYXNlcy4KCiogYGRiQ29ubmVjdGAgY3JlYXRlcyBhIGdlbmVyaWMgZGF0YSBiYXNlIGNvbm5lY3Rpb24gb2JqZWN0LgoqIGBkYkxpc3RUYWJsZXNgIHJldHVybnMgYSB2ZWN0b3Igd2l0aCB0aGUgbmFtZXMgb2YgdGhlIGF2YWlsYWJsZSB0YWJsZXMuCiogYGRiRGlzY29ubmVjdGAgY2xvc2VzIGEgZGF0YSBiYXNlIGNvbm5lY3Rpb24uCgpBbmFsb2dvdXMgcGFja2FnZXMgZXhpc3QgZm9yIFB5dGhvbiBhbmQgUGVybC4KClRoZSBmaXJzdCBzdGVwIGlzIHRvIGNvbm5lY3QgdG8gdGhlIGRhdGEgYmFzZS4KCjwhLS0gIyBub2xpbnQgc3RhcnQgLS0+CmBgYHtyLCBldmFsID0gRkFMU0V9CmxpYnJhcnkoZHBseXIpCmxpYnJhcnkoREJJKQpsaWJyYXJ5KFJTUUxpdGUpCmRiIDwtIGRiQ29ubmVjdChTUUxpdGUoKSwgIi9ncm91cC9zdGF0c29mdC9kYXRhL0RhdGFFeHBvMDkvb250aW1lLnNxbGl0ZTMiKQpgYGAKPCEtLSAjIG5vbGludCBlbmQgLS0+CgpUaGVyZSBpcyBvbmx5IG9uZSB0YWJsZToKYGBge3IsIGV2YWwgPSBGQUxTRX0KZGJMaXN0VGFibGVzKGRiKQojIyBbMV0gIm9udGltZSIKYGBgCgpgdGJsYCBjcmVhdGVzIGEgdGliYmxlLWxpa2Ugb2JqZWN0IGZvciBhIGRhdGEgYmFzZSB0YWJsZS4KCmBgYHtyLCBldmFsID0gRkFMU0V9CmZsIDwtIHRibChkYiwgIm9udGltZSIpCmBgYAoKVGhpcyBvYmplY3QgaXMgbm90IGEgcmVndWxhciB0aWJibGUsIGJ1dCBtYW55IHRpYmJsZSBvcGVyYXRpb25zIHdvcmsgb24gaXQuCgpUbyBmaW5kIHRoZSB2YXJpYWJsZXMgaW4gdGhlIHRhYmxlOgoKYGBge3IsIGV2YWwgPSBGQUxTRX0KdGJsX3ZhcnMoZmwpCiMjICBbMV0gIlllYXIiICAgICAgICAgICAgICAiTW9udGgiICAgICAgICAgICAgICJEYXlvZk1vbnRoIgojIyAgWzRdICJEYXlPZldlZWsiICAgICAgICAgIkRlcFRpbWUiICAgICAgICAgICAiQ1JTRGVwVGltZSIKIyMgIFs3XSAiQXJyVGltZSIgICAgICAgICAgICJDUlNBcnJUaW1lIiAgICAgICAgIlVuaXF1ZUNhcnJpZXIiCiMjIFsxMF0gIkZsaWdodE51bSIgICAgICAgICAiVGFpbE51bSIgICAgICAgICAgICJBY3R1YWxFbGFwc2VkVGltZSIKIyMgWzEzXSAiQ1JTRWxhcHNlZFRpbWUiICAgICJBaXJUaW1lIiAgICAgICAgICAgIkFyckRlbGF5IgojIyBbMTZdICJEZXBEZWxheSIgICAgICAgICAgIk9yaWdpbiIgICAgICAgICAgICAiRGVzdCIKIyMgWzE5XSAiRGlzdGFuY2UiICAgICAgICAgICJUYXhpSW4iICAgICAgICAgICAgIlRheGlPdXQiCiMjIFsyMl0gIkNhbmNlbGxlZCIgICAgICAgICAiQ2FuY2VsbGF0aW9uQ29kZSIgICJEaXZlcnRlZCIKIyMgWzI1XSAiQ2FycmllckRlbGF5IiAgICAgICJXZWF0aGVyRGVsYXkiICAgICAgIk5BU0RlbGF5IgojIyBbMjhdICJTZWN1cml0eURlbGF5IiAgICAgIkxhdGVBaXJjcmFmdERlbGF5IgpgYGAKCk1hbnkgYGRwbHlyYCBvcGVyYXRpb25zIGFyZSBzdXBwb3J0ZWQ6CgpgYGB7ciwgZXZhbCA9IEZBTFNFfQpzbSA8LSBzdW1tYXJpemUoZmwsIGxhc3RfeWVhciA9IG1heCh5ZWFyKSwgbiA9IG4oKSkKYGBgCgpUaGlzIGNvbXB1dGF0aW9uIGlzIGZhc3QgYmVjYXVzZSBpdCBkb2VzIG5vdCBhY3R1YWxseSBhY2Nlc3MKdGhlIGRhdGEgYmFzZS4KCkFuIG9wZXJhdGlvbiB0aGF0IG5lZWRzIHRoZSBkYXRhLCBzdWNoIGFzIHByaW50aW5nIHRoZSByZXN1bHQsIHdpbGwKYWNjZXNzIHRoZSBkYXRhIGJhc2UgYW5kIHRha2UgdGltZToKCmBgYHtyLCBldmFsID0gRkFMU0V9CnNtCiMjICMgU291cmNlOiAgIGxhenkgcXVlcnkgWz8/IHggMl0KIyMgIyBEYXRhYmFzZTogc3FsaXRlIDMuMzcuMgojIyAjICAgWy9tbnQvbmZzL2NsYXNuZXRhcHB2bS9ncm91cC9zdGF0c29mdC9kYXRhL0RhdGFFeHBvMDkvb250aW1lLnNxbGl0ZTNdCiMjICAgbGFzdF95ZWFyICAgICAgICAgbgojIyAgICAgICA8aW50PiAgICAgPGludD4KIyMgMSAgICAgIDIwMDggMTE4OTE0NDU4CmBgYAoKVGhlIGBzbWAgb2JqZWN0IGVzc2VudGlhbGx5IGNvbnRhaW5zIGEgU1FMIHF1ZXJ5LApbd2hpY2ggY2FuIGJlCmV4YW1pbmVkXShodHRwczovL2RicGx5ci50aWR5dmVyc2Uub3JnL2FydGljbGVzL3NxbC10cmFuc2xhdGlvbi5odG1sKQp1c2luZyBgc3FsX3JlbmRlcmAgZnJvbSB0aGUgYGRicGx5cmAgcGFja2FnZToKCmBgYHtyLCBldmFsID0gRkFMU0V9CmRicGx5cjo6c3FsX3JlbmRlcihzbSkKIyMgPFNRTD4gU0VMRUNUIE1BWChgeWVhcmApIEFTIGBsYXN0X3llYXJgLCBDT1VOVCgqKSBBUyBgbmAKIyMgRlJPTSBgb250aW1lYApgYGAKCkZsaWdodHMgdG8gQ0lEIGluIDIwMDg6CgpgYGB7ciwgZXZhbCA9IEZBTFNFfQpmbF9jaWQgPC0gZmlsdGVyKGZsLCB5ZWFyID09IDIwMDgsIERlc3QgPT0gIkNJRCIpCmRicGx5cjo6c3FsX3JlbmRlcihmbF9jaWQpCiMjIDxTUUw+IFNFTEVDVCAqCiMjIEZST00gYG9udGltZWAKIyMgV0hFUkUgKChgeWVhcmAgPSAyMDA4LjApIEFORCAoYERlc3RgID0gJ0NJRCcpKQpgYGAKCk51bWJlciBvZiBmbGlnaHRzIHRvIENJRCBpbiAyMDA4OgoKYGBge3IsIGV2YWwgPSBGQUxTRX0Kc20gPC0gc3VtbWFyaXplKGZsX2NpZCwgbiA9IG4oKSkKZGJwbHlyOjpzcWxfcmVuZGVyKHNtKQojIyA8U1FMPiBTRUxFQ1QgQ09VTlQoKikgQVMgYG5gCiMjIEZST00gYG9udGltZWAKIyMgV0hFUkUgKChgeWVhcmAgPSAyMDA4LjApIEFORCAoYERlc3RgID0gJ0NJRCcpKQpgYGAKCmBgYHtyLCBldmFsID0gRkFMU0V9CnNtCiMjICMgU291cmNlOiAgIGxhenkgcXVlcnkgWz8/IHggMV0KIyMgIyBEYXRhYmFzZTogc3FsaXRlIDMuMzcuMgojIyAjICAgWy9tbnQvbmZzL2NsYXNuZXRhcHB2bS9ncm91cC9zdGF0c29mdC9kYXRhL0RhdGFFeHBvMDkvb250aW1lLnNxbGl0ZTNdCiMjICAgICAgIG4KIyMgICA8aW50PgojIyAxICAyOTYxCmBgYAoKQnJlYWtkb3duIGJ5IHdoZXJlIHRoZSBmbGlnaHRzIG9yaWdpbmF0ZWQ6CgpgYGB7ciwgZXZhbCA9IEZBTFNFfQpzbSA8LSBncm91cF9ieShmbF9jaWQsIE9yaWdpbikgJT4lCiAgICBzdW1tYXJpemUobiA9IG4oKSkgJT4lCiAgICB1bmdyb3VwKCkKZGJwbHlyOjpzcWxfcmVuZGVyKHNtKQojIyA8U1FMPiBTRUxFQ1QgYE9yaWdpbmAsIENPVU5UKCopIEFTIGBuYAojIyBGUk9NIGBvbnRpbWVgCiMjIFdIRVJFICgoYHllYXJgID0gMjAwOC4wKSBBTkQgKGBEZXN0YCA9ICdDSUQnKSkKIyMgR1JPVVAgQlkgYE9yaWdpbmAKYGBgCgpgYGB7ciwgZXZhbCA9IEZBTFNFfQpzbQojIyAjIFNvdXJjZTogICBsYXp5IHF1ZXJ5IFs/PyB4IDJdCiMjICMgRGF0YWJhc2U6IHNxbGl0ZSAzLjM3LjIKIyMgIyAgIFsvbW50L25mcy9jbGFzbmV0YXBwdm0vZ3JvdXAvc3RhdHNvZnQvZGF0YS9EYXRhRXhwbzA5L29udGltZS5zcWxpdGUzXQojIyAgIE9yaWdpbiAgICAgbgojIyAgIDxjaHI+ICA8aW50PgojIyAxIEFUTCAgICAgIDExOQojIyAyIEJOQSAgICAgICAgMQojIyAzIENWRyAgICAgIDI5MgojIyA0IERFTiAgICAgIDIzMAojIyA1IERGVyAgICAgIDUxOQojIyA2IERUVyAgICAgIDM0NQojIyA3IE1TUCAgICAgIDI0MQojIyA4IE9SRCAgICAgMTIxNApgYGAKCk1hbnkgYmFzZSBSIGZ1bmN0aW9ucyBjYW4gYmUgY29udmVydGVkIHRvIFNRTDoKCmBgYHtyLCBldmFsID0gRkFMU0V9CnNtIDwtIGdyb3VwX2J5KGZsX2NpZCwgT3JpZ2luKSAlPiUKICAgIHN1bW1hcml6ZShhdmVfYXJyX2RlbF9wb3MgPSBtZWFuKHBtYXgoQXJyRGVsYXksIDApLCBuYS5ybSA9IFRSVUUpKSAlPiUKICAgIHVuZ3JvdXAoKQpkYnBseXI6OnNxbF9yZW5kZXIoc20pCiMjIDxTUUw+IFNFTEVDVCBgT3JpZ2luYCwgQVZHKE1BWChgQXJyRGVsYXlgLCAwLjApKSBBUyBgYXZlX2Fycl9kZWxfcG9zYAojIyBGUk9NIGBvbnRpbWVgCiMjIFdIRVJFICgoYHllYXJgID0gMjAwOC4wKSBBTkQgKGBEZXN0YCA9ICdDSUQnKSkKIyMgR1JPVVAgQlkgYE9yaWdpbmAKYGBgCgpgYGB7ciwgZXZhbCA9IEZBTFNFfQpzbV9jaWQgPC0gc20Kc20KIyMgIyBTb3VyY2U6ICAgbGF6eSBxdWVyeSBbPz8geCAyXQojIyAjIERhdGFiYXNlOiBzcWxpdGUgMy4zNy4yCiMjICMgICBbL21udC9uZnMvY2xhc25ldGFwcHZtL2dyb3VwL3N0YXRzb2Z0L2RhdGEvRGF0YUV4cG8wOS9vbnRpbWUuc3FsaXRlM10KIyMgICBPcmlnaW4gYXZlX2Fycl9kZWxfcG9zCiMjICAgPGNocj4gICAgICAgICAgICA8ZGJsPgojIyAxIEFUTCAgICAgICAgICAgICAgIDE3LjQKIyMgMiBCTkEgICAgICAgICAgICAgICAxOAojIyAzIENWRyAgICAgICAgICAgICAgIDEyLjQKIyMgNCBERU4gICAgICAgICAgICAgICAxMi4xCiMjIDUgREZXICAgICAgICAgICAgICAgMTEuMwojIyA2IERUVyAgICAgICAgICAgICAgIDE1LjAKIyMgNyBNU1AgICAgICAgICAgICAgICAyMC4xCiMjIDggT1JEICAgICAgICAgICAgICAgMjEuNgpgYGAKCmBtdXRhdGUoKWAgY2FuIGFsc28gYmUgdXNlZDoKCjwhLS0gIyMgbm9saW50IHN0YXJ0OiBsaW5lX2xlbmd0aCAtLT4KYGBge3IsIGV2YWwgPSBGQUxTRX0KZmxfY2lkX2dhaW4gPC0gbXV0YXRlKGZsX2NpZCwgR2FpbiA9IERlcERlbGF5IC0gQXJyRGVsYXkpCmRicGx5cjo6c3FsX3JlbmRlcihmbF9jaWRfZ2FpbikKIyMgPFNRTD4gU0VMRUNUIGBZZWFyYCwgYE1vbnRoYCwgYERheW9mTW9udGhgLCBgRGF5T2ZXZWVrYCwgYERlcFRpbWVgLCBgQ1JTRGVwVGltZWAsIGBBcnJUaW1lYCwgYENSU0FyclRpbWVgLCBgVW5pcXVlQ2FycmllcmAsIGBGbGlnaHROdW1gLCBgVGFpbE51bWAsIGBBY3R1YWxFbGFwc2VkVGltZWAsIGBDUlNFbGFwc2VkVGltZWAsIGBBaXJUaW1lYCwgYEFyckRlbGF5YCwgYERlcERlbGF5YCwgYE9yaWdpbmAsIGBEZXN0YCwgYERpc3RhbmNlYCwgYFRheGlJbmAsIGBUYXhpT3V0YCwgYENhbmNlbGxlZGAsIGBDYW5jZWxsYXRpb25Db2RlYCwgYERpdmVydGVkYCwgYENhcnJpZXJEZWxheWAsIGBXZWF0aGVyRGVsYXlgLCBgTkFTRGVsYXlgLCBgU2VjdXJpdHlEZWxheWAsIGBMYXRlQWlyY3JhZnREZWxheWAsIGBEZXBEZWxheWAgLSBgQXJyRGVsYXlgIEFTIGBHYWluYAojIyBGUk9NIGBvbnRpbWVgCiMjIFdIRVJFICgoYHllYXJgID0gMjAwOC4wKSBBTkQgKGBEZXN0YCA9ICdDSUQnKSkKYGBgCjwhLS0gIyMgbm9saW50IGVuZCAtLT4KCmBgYHtyLCBldmFsID0gRkFMU0V9Cmdyb3VwX2J5KGZsX2NpZF9nYWluLCBPcmlnaW4pICU+JQogICAgc3VtbWFyaXplKG1lYW5fZ2FpbiA9IG1lYW4oR2FpbikpICU+JQogICAgdW5ncm91cCgpCiMjICMgU291cmNlOiAgIGxhenkgcXVlcnkgWz8/IHggMl0KIyMgIyBEYXRhYmFzZTogc3FsaXRlIDMuMzcuMgojIyAjICAgWy9tbnQvbmZzL2NsYXNuZXRhcHB2bS9ncm91cC9zdGF0c29mdC9kYXRhL0RhdGFFeHBvMDkvb250aW1lLnNxbGl0ZTNdCiMjICAgT3JpZ2luIG1lYW5fZ2FpbgojIyAgIDxjaHI+ICAgICAgPGRibD4KIyMgMSBBVEwgICAgICAgIDguMjQKIyMgMiBCTkEgICAgICAtMjYKIyMgMyBDVkcgICAgICAgLTEuNjAKIyMgNCBERU4gICAgICAgIDAuMjc4CiMjIDUgREZXICAgICAgICA0LjEyCiMjIDYgRFRXICAgICAgICAzLjA0CiMjIDcgTVNQICAgICAgIC0wLjM1NwojIyA4IE9SRCAgICAgICAtMS4zNgpgYGAKCk9uY2UgYSBmaWx0ZXJlZCByZXN1bHQgaXMgc21hbGwgZW5vdWdoIGl0IGNhbiBiZSBicm91Z2h0IG92ZXIgYXMgYQpyZWd1bGFyIHRpYmJsZSB1c2luZwoKYGBge3IsIGV2YWwgPSBGQUxTRX0KdGIgPC0gY29sbGVjdChmbF9jaWQpCmBgYAoKVGhlcmUgYXJlIHNvbWUgbGltaXRhdGlvbnMgdG8gb3BlcmF0aW9ucyB0aGF0IGNhbiBiZSBkb25lIG9uIHRhYmxlcyBpbgpkYXRhIGJhc2VzLgoKRGVmaW5pbmcgYW5kIHVzaW5nIGEgZnVuY3Rpb24gd29ya3MgZm9yIGEgdGliYmxlOgoKYGBge3IsIGV2YWwgPSBGQUxTRX0KcG9zX21lYW4gPC0gZnVuY3Rpb24oeCkgbWVhbihwbWF4KHgsIDApKQpncm91cF9ieSh0YiwgT3JpZ2luKSAlPiUKICAgIHN1bW1hcml6ZShhdmVfYXJyX2RlbF9wb3MgPSBwb3NfbWVhbihBcnJEZWxheSkpICU+JQogICAgdW5ncm91cCgpCmBgYAoKQnV0IHRoaXMgZmFpbHM6CgpgYGB7ciwgZXZhbCA9IEZBTFNFfQpncm91cF9ieShmbF9jaWQsIE9yaWdpbikgJT4lCiAgICBzdW1tYXJpemUoYXZlX2Fycl9kZWxfcG9zID0gcG9zX21lYW4oQXJyRGVsYXkpKSAlPiUKICAgIHVuZ3JvdXAoKQojIyBFcnJvcjogbm8gc3VjaCBmdW5jdGlvbjogcG9zX21lYW4KYGBgCgpTb21lIG9wZXJhdGlvbnMgbWF5IHJlcXVpcmUgc29tZSBtb2RpZmljYXRpb25zLgoKUmVjb21wdXRpbmcgYHNtX2NpZGA6CgpgYGB7ciwgZXZhbCA9IEZBTFNFfQpmbCA8LSB0YmwoZGIsICJvbnRpbWUiKQpmbF9jaWQgPC0gZmlsdGVyKGZsLCB5ZWFyID09IDIwMDgsIERlc3QgPT0gIkNJRCIpCnNtX2NpZCA8LSBncm91cF9ieShmbF9jaWQsIE9yaWdpbikgJT4lCiAgICBzdW1tYXJpemUoYXZlX2Fycl9kZWxfcG9zID0gbWVhbihwbWF4KEFyckRlbGF5LCAwKSwgbmEucm0gPSBUUlVFKSkgJT4lCiAgICB1bmdyb3VwKCkKYGBgCgpKb2luaW5nIHdpdGggYSBsb2NhbCB0YWJsZSBkb2VzIG5vdCB3b3JrOgoKYGBge3IsIGV2YWwgPSBGQUxTRX0KYXAgPC0gc2VsZWN0KG55Y2ZsaWdodHMxMzo6YWlycG9ydHMsIGZhYSwgbmFtZSkKc21fY2lkIDwtIGxlZnRfam9pbihzbV9jaWQsIGFwLCBjKE9yaWdpbiA9ICJmYWEiKSkKIyMgRXJyb3IgaW4gYGF1dG9fY29weSgpYDoKIyMgISBgeGAgYW5kIGB5YCBtdXN0IHNoYXJlIHRoZSBzYW1lIHNyYy4KIyMg4oS5IHNldCBgY29weWAgPSBUUlVFIChtYXkgYmUgc2xvdykuCmBgYAoKQnV0IHRoaXMgZG9lcyB3b3JrOgoKYGBge3IsIGV2YWwgPSBGQUxTRX0Kc21fY2lkIDwtIGxlZnRfam9pbihzbV9jaWQsIGFwLCBjKE9yaWdpbiA9ICJmYWEiKSwgY29weSA9IFRSVUUpCmBgYAoKVGhpcyBjb3BpZXMgdGhlIGBhcGAgdGFibGUgdG8gdGhlIGRhdGEgYmFzZToKCmBgYHtyLCBldmFsID0gRkFMU0V9CmRiTGlzdFRhYmxlcyhkYikKIyMgWzFdICJkYnBseXJfMDAxIiAgICJvbnRpbWUiICAgICAgICJzcWxpdGVfc3RhdDEiICJzcWxpdGVfc3RhdDQiCmBgYAoKRmluaXNoIGJ5IGRpc2Nvbm5lY3RpbmcgZnJvbSB0aGUgZGF0YSBiYXNlOgoKYGBge3IsIGV2YWwgPSBGQUxTRX0KZGJEaXNjb25uZWN0KGRiKQpgYGAKClRoaXMgaXMgbm90IG5lY2Vzc2FyeSBmb3IgU1FMaXRlIGJ1dCBpcyBmb3Igb3RoZXIgZGF0YSBiYXNlcywgc28gaXQgaXMKZ29vZCBwcmFjdGljZS4KClNvbWUgbm90ZXM6CgoqIEJ1aWxkaW5nIGRhdGEgYmFzZSBxdWVyaWVzIHdpdGggYGRwbHlyYCBpcyB2ZXJ5IGVmZmljaWVudC4KCiogUnVubmluZyB0aGUgcXVlcmllcyBjYW4gdGFrZSB0aW1lLgoKKiBUcmFuc2ZlcnJpbmcgKHBhcnRpYWwpIHJlc3VsdHMgdG8gUiBjYW4gdGFrZSB0aW1lLgoKKiBJbmFkdmVydGVudGx5IGFza2luZyB0byB0cmFuc2ZlciBhIHJlc3VsdCB0aGF0IGlzIHRvbyBsYXJnZQogIGNvdWxkIGNhdXNlIHByb2JsZW1zLgoKKiBPdGhlciBkYXRhIGJhc2VzIG1heSBhcnJhbmdlIHRvIGNhY2hlIHF1ZXJ5IHJlc3VsdHMuCgoqIE1vc3QgZGF0YSBiYXNlcywgaW5jbHVkaW5nIFNRTGl0ZSwgc3VwcG9ydCBjcmVhdGluZyBfaW5kaWNlc18gdGhhdAogIHNwZWVkIHVwIGNlcnRhaW4gcXVlcmllcyBhdCB0aGUgZXhwZW5zZSBvZiB1c2luZyBtb3JlIHN0b3JhZ2UuCgpUaGUgW2BkYnBseXJgIHdlYiBzaXRlXShodHRwczovL2RicGx5ci50aWR5dmVyc2Uub3JnL2luZGV4Lmh0bWwpCnByb3ZpZGVzIG1vcmUgaW5mb3JtYXRpb24gb24gdXNpbmcgZGF0YSBiYXNlcyB3aXRoIGBkcGx5cmAuCgoKIyMgT3RoZXIgV3JhbmdsaW5nIFRvb2xzCgpTb21lIGFkZGl0aW9uYWwgdG9vbHMgcHJvdmlkZWQgYnkgYHRpZHlyYDoKCiogYHNlcGFyYXRlKClgOiBzcGxpdCBjaGFyYWN0ZXIgdmFyaWFibGUgaW50byBzZXZlcmFsIHZhcmlhYmxlczsKCiogYGZpbGwoKWA6IGZpbGwgaW4gbWlzc2luZyB2YWx1ZXMgaW4gYSB2YXJpYWJsZSAoZS5nLiBjYXJyeSBmb3J3YXJkKTsKCiogYGNvbXBsZXRlKClgOiBhZGQgbWlzc2luZyBjYXRlZ29yeSBjbGFzc2VzIChlLmcuIHplcm8gY291bnRzKS4KCldlIGhhdmUgYWxyZWFkeSBzZWVuIGBwaXZvdF9sb25nZXIoKWAgYW5kIGBwaXZvdF93aWRlcigpYC4KCgojIyMgU2VwYXJhdGluZyBWYXJpYWJsZXMKCkEgZGF0YSBzZXQgZnJvbSB0aGUgV0hPIG9uIHR1YmVyY3Vsb3NpcyBjYXNlczoKCmBgYHtyfQpoZWFkKHdobywgMikKYGBgCgpTb21lIG9mIHRoZSByZW1haW5pbmcgdmFyaWFibGUgbmFtZXM6CgpgYGB7cn0KbmFtZXMod2hvKSAlPiUgdGFpbCgyMCkKYGBgCgpUaGUgdmFyaWFibGUgYG5ld19zcF9tMjUzNGAsIGZvciBleGFtcGxlLCByZXByZXNlbnRzIHRoZSBudW1iZXIgb2YgY2FzZXMgZm9yCgoqIHR5cGUgYHNwYCAocG9zaXRpdmUgcHVsbW9uYXJ5IHNtZWFyKTsKKiBzZXggYG1gOwoqIGFnZSBiZXR3ZWVuIDI1IGFuZCAzNC4KCkl0IHdvdWxkIGJlIGJldHRlciAodGlkaWVyKSB0byBoYXZlIHNlcGFyYXRlIHZhcmlhYmxlcyBmb3IKCiogY291bnQKKiB0eXBlCiogc2V4CiogbG93ZXIgYm91bmQgb24gYWdlIGJyYWNrZXQKKiB1cHBlciBib3VuZCBvbiBhZ2UgYnJhY2tldAoKQSBwaXBlbGluZSB0byBjbGVhbiB0aGlzIHVwIGludm9sdmVzIHBpdm90aW5nIHRvIGxvbmcgZm9ybSBhbmQgc2V2ZXJhbApgc2VwYXJhdGVgIGFuZCBgbXV0YXRlYCBzdGVwcy4KClRoZSBvcmlnaW5hbCBkYXRhIGZyYW1lOgoKYGBge3J9CndobwpgYGAKCkZpcnN0IHBpdm90IHRvIGxvbmdlciBmb3JtOgoKYGBge3J9Cndob19jbGVhbiA8LQogICAgcGl2b3RfbG9uZ2VyKHdobywKICAgICAgICAgICAgICAgICBuZXdfc3BfbTAxNCA6IG5ld3JlbF9mNjUsCiAgICAgICAgICAgICAgICAgbmFtZXNfdG8gPSAia2V5IiwKICAgICAgICAgICAgICAgICB2YWx1ZXNfdG8gPSAiY291bnQiKQpgYGAKYGBge3J9Cndob19jbGVhbgpgYGAKClJlbW92ZSBgIm5ldyJgIG9yIGAibmV3XyJgIHByZWZpeDoKCmBgYHtyfQp3aG9fY2xlYW4gPC0KICAgIHBpdm90X2xvbmdlcih3aG8sCiAgICAgICAgICAgICAgICAgbmV3X3NwX20wMTQgOiBuZXdyZWxfZjY1LAogICAgICAgICAgICAgICAgIG5hbWVzX3RvID0gImtleSIsCiAgICAgICAgICAgICAgICAgdmFsdWVzX3RvID0gImNvdW50IikgJT4lCiAgICBtdXRhdGUoa2V5ID0gc3ViKCJuZXdfPyIsICIiLCBrZXkpKQpgYGAKYGBge3J9Cndob19jbGVhbgpgYGAKClNlcGFyYXRlIGBrZXlgIGludG8gYHR5cGVgIGFuZCBgc2V4YWdlYDoKCmBgYHtyfQp3aG9fY2xlYW4gPC0KICAgIHBpdm90X2xvbmdlcih3aG8sCiAgICAgICAgICAgICAgICAgbmV3X3NwX20wMTQgOiBuZXdyZWxfZjY1LAogICAgICAgICAgICAgICAgIG5hbWVzX3RvID0gImtleSIsCiAgICAgICAgICAgICAgICAgdmFsdWVzX3RvID0gImNvdW50IikgJT4lCiAgICBtdXRhdGUoa2V5ID0gc3ViKCJuZXdfPyIsICIiLCBrZXkpKSAlPiUKICAgIHNlcGFyYXRlKGtleSwgYygidHlwZSIsICJzZXhhZ2UiKSkKYGBgCmBgYHtyfQp3aG9fY2xlYW4KYGBgCgpTZXBhcmF0ZSBgc2V4YWdlYCBpbnRvIGBzZXhgIGFuZCBgYWdlYDoKCmBgYHtyfQp3aG9fY2xlYW4gPC0KICAgIHBpdm90X2xvbmdlcih3aG8sCiAgICAgICAgICAgICAgICAgbmV3X3NwX20wMTQgOiBuZXdyZWxfZjY1LAogICAgICAgICAgICAgICAgIG5hbWVzX3RvID0gImtleSIsCiAgICAgICAgICAgICAgICAgdmFsdWVzX3RvID0gImNvdW50IikgJT4lCiAgICBtdXRhdGUoa2V5ID0gc3ViKCJuZXdfPyIsICIiLCBrZXkpKSAlPiUKICAgIHNlcGFyYXRlKGtleSwgYygidHlwZSIsICJzZXhhZ2UiKSkgJT4lCiAgICBzZXBhcmF0ZShzZXhhZ2UsIGMoInNleCIsICJhZ2UiKSwKICAgICAgICAgICAgIHNlcCA9IDEpCmBgYApgYGB7cn0Kd2hvX2NsZWFuCmBgYAoKRml4IHVwIGFnZSBjYXRlZ29yaWVzOgoKYGBge3J9Cndob19jbGVhbiA8LQogICAgcGl2b3RfbG9uZ2VyKHdobywKICAgICAgICAgICAgICAgICBuZXdfc3BfbTAxNCA6IG5ld3JlbF9mNjUsCiAgICAgICAgICAgICAgICAgbmFtZXNfdG8gPSAia2V5IiwKICAgICAgICAgICAgICAgICB2YWx1ZXNfdG8gPSAiY291bnQiKSAlPiUKICAgIG11dGF0ZShrZXkgPSBzdWIoIm5ld18/IiwgIiIsIGtleSkpICU+JQogICAgc2VwYXJhdGUoa2V5LCBjKCJ0eXBlIiwgInNleGFnZSIpKSAlPiUKICAgIHNlcGFyYXRlKHNleGFnZSwgYygic2V4IiwgImFnZSIpLAogICAgICAgICAgICAgc2VwID0gMSkgJT4lCiAgICBtdXRhdGUoYWdlID0gc3ViKCIwMTQiLCAiMDAxNCIsIGFnZSkpICU+JQogICAgbXV0YXRlKGFnZSA9IHN1YigiNjUiLCAiNjVJbmYiLCBhZ2UpKQpgYGAKYGBge3J9Cndob19jbGVhbgpgYGAKCkZpbmFsbHksIHNwbGl0IGBhZ2VgIGludG8gYGFnZV9sb2AgYW5kIGBhZ2UtaGlgOgoKYGBge3J9Cndob19jbGVhbiA8LQogICAgcGl2b3RfbG9uZ2VyKHdobywKICAgICAgICAgICAgICAgICBuZXdfc3BfbTAxNCA6IG5ld3JlbF9mNjUsCiAgICAgICAgICAgICAgICAgbmFtZXNfdG8gPSAia2V5IiwKICAgICAgICAgICAgICAgICB2YWx1ZXNfdG8gPSAiY291bnQiKSAlPiUKICAgIG11dGF0ZShrZXkgPSBzdWIoIm5ld18/IiwgIiIsIGtleSkpICU+JQogICAgc2VwYXJhdGUoa2V5LCBjKCJ0eXBlIiwgInNleGFnZSIpKSAlPiUKICAgIHNlcGFyYXRlKHNleGFnZSwgYygic2V4IiwgImFnZSIpLAogICAgICAgICAgICAgc2VwID0gMSkgJT4lCiAgICBtdXRhdGUoYWdlID0gc3ViKCIwMTQiLCAiMDAxNCIsIGFnZSkpICU+JQogICAgbXV0YXRlKGFnZSA9IHN1YigiNjUiLCAiNjVJbmYiLCBhZ2UpKSAlPiUKICAgIHNlcGFyYXRlKGFnZSwgYygiYWdlX2xvIiwgImFnZV9oaSIpLAogICAgICAgICAgICAgc2VwID0gMiwKICAgICAgICAgICAgIGNvbnZlcnQgPSBUUlVFKQpgYGAKYGBge3J9Cndob19jbGVhbgpgYGAKCkEgcGxvdCBvZiB0b3RhbCBudW1iZXJzIG9mIGNhc2VzIGZvciBhIGZldyBjb3VudHJpZXMgb3ZlciB0aW1lOgoKYGBge3Igd2hvLXBsb3QsIGV2YWwgPSBGQUxTRX0KZmlsdGVyKHdob19jbGVhbiwKICAgICAgIGlzbzMgJWluJSBjKCJDQU4iLCAiREVVIiwgIkdCUiIsICJVU0EiKSkgJT4lCiAgICBncm91cF9ieSh5ZWFyLCBpc28zKSAlPiUKICAgIHN1bW1hcml6ZShjb3VudCA9IHN1bShjb3VudCwgbmEucm0gPSBUUlVFKSkgJT4lCiAgICB1bmdyb3VwKCkgJT4lCiAgICBnZ3Bsb3QoKSArCiAgICBnZW9tX2xpbmUoYWVzKHggPSB5ZWFyLAogICAgICAgICAgICAgICAgICB5ID0gY291bnQsCiAgICAgICAgICAgICAgICAgIGNvbG9yID0gaXNvMykpICsKICAgIHRobQpgYGAKCmBgYHtyIHdoby1wbG90LCBlY2hvID0gRkFMU0V9CmBgYAoKCiMjIyBGaWxsaW5nIEluIE9taXR0ZWQgVmFsdWVzCgpUaGUgW1JlZnVnZWUgUHJvY2Vzc2luZyBDZW50ZXJdKGh0dHA6Ly93d3cud3JhcHNuZXQub3JnLykgcHJvdmlkZXMKaW5mb3JtYXRpb24gZnJvbSB0aGUgVW5pdGVkIFN0YXRlcyBSZWZ1Z2VlIEFkbWlzc2lvbnMgUHJvZ3JhbSAoVVNSQVApLAppbmNsdWRpbmcKW2Fycml2YWxzIGJ5IHN0YXRlIGFuZCBuYXRpb25hbGl0eV0oaHR0cDovL3d3dy53cmFwc25ldC5vcmcvYWRtaXNzaW9ucy1hbmQtYXJyaXZhbHMvKS4KClRocmVlIGZpbGVzIGFyZSBhdmFpbGFibGUgbG9jYWxseToKCiogZGF0YSBmcm9tIFtlYXJseSBKYW51YXJ5CiAgMjAxN10oaHR0cDovL2hvbWVwYWdlLnN0YXQudWlvd2EuZWR1L35sdWtlL2RhdGEvQXJyaXZhbHMtMjAxNy0wMS0wNi54bHMpOwoKKiBkYXRhIGZyb20gW2Vhcmx5IEFwcmlsCiAgMjAxN10oaHR0cDovL2hvbWVwYWdlLnN0YXQudWlvd2EuZWR1L35sdWtlL2RhdGEvQXJyaXZhbHMtMjAxNy0wNC0wNS54bHMpLgoKKiBkYXRhIGZyb20gW2Vhcmx5IE1hcmNoCiAgMjAxOF0oaHR0cDovL2hvbWVwYWdlLnN0YXQudWlvd2EuZWR1L35sdWtlL2RhdGEvQXJyaXZhbHMtMjAxOC0wMy0wNS54bHMpLgoKVGhlc2UgYXJlIEV4Y2VsIHNwcmVhZCBzaGVldHMuCgpSZWFkaW5nIGluIHRoZSB0aGlyZCBmaWxlLCB3aXRoIGRhdGEgZm9yIE9jdG9iZXIgMjAxNyB0aHJvdWdoIEZlYnJ1YXJ5CjIwMTg6CgpgYGB7ciwgbWVzc2FnZSA9IEZBTFNFfQpyZWZfdXJsIDwtCiAgICAiaHR0cDovL2hvbWVwYWdlLnN0YXQudWlvd2EuZWR1L35sdWtlL2RhdGEvQXJyaXZhbHMtMjAxOC0wMy0wNS54bHMiCnJlZl9maWxlIDwtICJBcnJpdmFscy0yMDE4LTAzLTA1LnhscyIKaWYgKCEgZmlsZS5leGlzdHMocmVmX2ZpbGUpKQogICAgZG93bmxvYWQuZmlsZShyZWZfdXJsLCByZWZfZmlsZSkKCmxpYnJhcnkocmVhZHhsKQpyZWYgPC0gcmVhZF9leGNlbChyZWZfZmlsZSwgc2tpcCA9IDE2KSAjIyBza2lwIGhlYWRlcgpyZWYgPC0gaGVhZChyZWYsIC0yKSAjIyBkcm9wIGxhc3QgdHdvIHJvd3MKbmFtZXMocmVmKVsxXSA8LSAiRGVzdGluYXRpb24iCmBgYAoKVGhlIGBEZXN0aW5hdGlvbmAgY29sdW1uIG5lZWRzIGZpbGxpbmcgaW4gKHRoaXMgaXMgY29tbW9uIGluCnNwcmVhZCBzaGVldCBkYXRhKToKCmBgYHtyfQpyZWYKYGBgCgpGaWxsIGRvd24gdGhlIERlc3RpbmF0aW9uOgoKYGBge3J9CnJlZl9jbGVhbiA8LQogICAgZmlsbChyZWYsIERlc3RpbmF0aW9uKQpgYGAKCmBgYHtyfQpyZWZfY2xlYW4KYGBgCgpgYGB7ciwgaW5jbHVkZSA9IEZBTFNFfQojIyBjaGVjayB0aGUgdG90YWxzCnJlZi50b3QgPC0gZmlsdGVyKHJlZl9jbGVhbiwgaXMubmEoTmF0aW9uYWxpdHkpKSAlPiUKICAgIHNlbGVjdChEZXN0aW5hdGlvbiwgT2N0IDogU2VwKQpydCA8LSBmaWx0ZXIocmVmX2NsZWFuLCAhIGlzLm5hKE5hdGlvbmFsaXR5KSkgJT4lCiAgICBzZWxlY3QoRGVzdGluYXRpb24sIE5hdGlvbmFsaXR5LCBPY3QgOiBTZXApICU+JQogICAgZ3JvdXBfYnkoRGVzdGluYXRpb24pICU+JQogICAgc3VtbWFyaXplX2lmKGlzLm51bWVyaWMsIHN1bSkKc3RvcGlmbm90KGlkZW50aWNhbChydCwgcmVmLnRvdCkpCmBgYAoKRHJvcCB0aGUgdG90YWxzOgoKYGBge3J9CnJlZl9jbGVhbiA8LQogICAgZmlsbChyZWYsIERlc3RpbmF0aW9uKSAlPiUKICAgIGZpbHRlcighIGlzLm5hKE5hdGlvbmFsaXR5KSkKYGBgCgpgYGB7cn0KcmVmX2NsZWFuCmBgYAoKUGl2b3QgdG8gYSB0aWRpZXIgZm9ybSB3aXRoIG9uZSByb3cgcGVyIG1vbnRoOgoKYGBge3J9CnJlZl9jbGVhbiA8LQogICAgZmlsbChyZWYsIERlc3RpbmF0aW9uKSAlPiUKICAgIGZpbHRlcighIGlzLm5hKE5hdGlvbmFsaXR5KSkgJT4lCiAgICBwaXZvdF9sb25nZXIoCiAgICAgICAgT2N0IDogU2VwLAogICAgICAgIG5hbWVzX3RvID0gIm1vbnRoIiwKICAgICAgICB2YWx1ZXNfdG8gPSAiY291bnQiKQpgYGAKCmBgYHtyfQpyZWZfY2xlYW4KYGBgCgpUb3AgNSBkZXN0aW5hdGlvbiBzdGF0ZXM6CgpgYGB7cn0KZ3JvdXBfYnkocmVmX2NsZWFuLCBEZXN0aW5hdGlvbikgJT4lCiAgICBzdW1tYXJpemUoY291bnQgPSBzdW0oY291bnQpKSAlPiUKICAgIHVuZ3JvdXAoKSAlPiUKICAgIHNsaWNlX21heChjb3VudCwgbiA9IDUpCmBgYAoKVG9wIDUgb3JpZ2luIG5hdGlvbmFsaXRpZXM6CgpgYGB7cn0KZ3JvdXBfYnkocmVmX2NsZWFuLCBOYXRpb25hbGl0eSkgJT4lCiAgICBzdW1tYXJpemUoY291bnQgPSBzdW0oY291bnQpKSAlPiUKICAgIHVuZ3JvdXAoKSAlPiUKICAgIHNsaWNlX21heChjb3VudCwgbiA9IDUpCmBgYAoKCiMjIyBDb21wbGV0aW5nIE1pc3NpbmcgRmFjdG9yIExldmVsIENvbWJpbmF0aW9ucwoKVGhpcyBleGFtcGxlIGlzIGJhc2VkIG9uIGEgW2Jsb2cKcG9zdF0oaHR0cHM6Ly93aW4tdmVjdG9yLmNvbS8yMDE3LzAyLzIxL3RoZS16ZXJvLWJ1Zy8pIHVzaW5nIGRhdGEKZnJvbSB0aGUgW1NpZ25pZmljYW50IEVhcnRocXVha2UKRGF0YWJhc2VdKGh0dHBzOi8vd3d3Lm5jZWkubm9hYS5nb3YvYWNjZXNzL21ldGFkYXRhL2xhbmRpbmctcGFnZS9iaW4vaXNvP2lkPWdvdi5ub2FhLm5nZGMubWdnLmhhemFyZHM6RzAxMjE1MykuCjwhLS1odHRwczovL3d3dy5uZ2RjLm5vYWEuZ292L2hhemVsL3ZpZXcvaGF6YXJkcy9lYXJ0aHF1YWtlL3NlYXJjaC0tPgoKVGhlIGRhdGEgdXNlZCBpcyBhIHNlbGVjdGlvbiBmb3IgTm9ydGggQW1lcmljYSBhbmQgSGF3YWlpLgoKUmVhZGluZyBpbiB0aGUgZGF0YToKCmBgYHtyfQppZiAoISBmaWxlLmV4aXN0cygiZWFydGhxdWFrZXMudHN2IikpCiAgICBkb3dubG9hZC5maWxlKCJodHRwOi8vaG9tZXBhZ2Uuc3RhdC51aW93YS5lZHUvfmx1a2UvZGF0YS9lYXJ0aHF1YWtlcy50c3YiLAogICAgICAgICAgICAgICAgICAiZWFydGhxdWFrZXMudHN2IikKZXEgPC0gcmVhZC50YWJsZSgiZWFydGhxdWFrZXMudHN2IiwKICAgICAgICAgICAgICAgICBoZWFkZXIgPSBUUlVFLAogICAgICAgICAgICAgICAgIHNlcCA9ICJcdCIpCmVxIDwtIGFzX3RpYmJsZShlcSkKYGBgCgpgYGB7cn0Kc2VsZWN0KGVxLCBTVEFURSwgZXZlcnl0aGluZygpKQpgYGAKClNvbWUgYFNUQVRFYCBlbnRyaWVzIGFyZSBibGFuaywgYnV0IHRoZSBsb2NhdGlvbiBuYW1lIGlkZW50aWZpZXMgdGhlIHN0YXRlOgoKYGBge3J9CmZpbHRlcihlcSwgU1RBVEUgPT0gIiIpICU+JQogICAgc2VsZWN0KFNUQVRFLCBMT0NBVElPTl9OQU1FKQpgYGAKCk9uZSB3YXkgdG8gZml4IHRoaXM6CgpgYGB7cn0KYmFkaWR4IDwtIHdoaWNoKGVxJFNUQVRFID09ICIiKQpiYWRzdGF0ZSA8LSBzdWIoIjouKiIsICIiLAogICAgICAgICAgICAgICAgZXEkTE9DQVRJT05fTkFNRVtiYWRpZHhdKQplcSRTVEFURVtiYWRpZHhdIDwtCiAgICBzdGF0ZS5hYmJbbWF0Y2godG9sb3dlcihiYWRzdGF0ZSksCiAgICAgICAgICAgICAgICAgICAgdG9sb3dlcihzdGF0ZS5uYW1lKSldCmBgYAoKQWRkIGZ1bGwgc3RhdGUgbmFtZXMgYXMgYSBmYWN0b3I6CgpgYGB7cn0Kc3RhdGVzIDwtIGRhdGEuZnJhbWUoU1RBVEUgPSBzdGF0ZS5hYmIsCiAgICAgICAgICAgICAgICAgICAgIFNUQVRFX05BTUUgPSBzdGF0ZS5uYW1lKQplcSA8LSBsZWZ0X2pvaW4oZXEsIHN0YXRlcywgIlNUQVRFIikKZXEgPC0gbXV0YXRlKGVxLAogICAgICAgICAgICAgU1RBVEVfTkFNRSA9IGZhY3RvcihTVEFURV9OQU1FLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdGF0ZS5uYW1lKSkKYGBgCgpMb29rIGF0IHRvdGFsIG51bWJlciBvZiBlYXJ0aHF1YWtlcyBieSBzdGF0ZToKCmBgYHtyfQp0YmwgPC0gY291bnQoZXEsIFNUQVRFX05BTUUpCmhlYWQodGJsLCA2KQpgYGAKCmBgYHtyLCBmaWcuaGVpZ2h0ID0gNSwgZmlnLndpZHRoID0gOCwgY2xhc3Muc291cmNlID0gImZvbGQtaGlkZSJ9CmxpYnJhcnkoZm9yY2F0cykKcGIgPC0gZ2dwbG90KHRibCwgYWVzKHkgPSBmY3RfcmV2KFNUQVRFX05BTUUpLCB4ID0gbikpICsKICAgIGdlb21fY29sKCkgKwogICAgdGhtICsKICAgIHRoZW1lKGF4aXMudGl0bGUueSA9IGVsZW1lbnRfYmxhbmsoKSkKcHMgPC0gZ2dwbG90KHRibCwgYWVzKHkgPSBmY3RfcmV2KFNUQVRFX05BTUUpLCB4ID0gbikpICsKICAgIGdlb21fcG9pbnQoKSArCiAgICBnZW9tX3NlZ21lbnQoYWVzKHhlbmQgPSAwLCB5ZW5kID0gZmN0X3JldihTVEFURV9OQU1FKSkpICsKICAgIHRobSArCiAgICB0aGVtZShheGlzLnRpdGxlLnkgPSBlbGVtZW50X2JsYW5rKCkpCnBiIHwgcHMKYGBgCgpNYW55IHN0YXRlcywgaW5jbHVkaW5nIElvd2EsIGFyZSBtaXNzaW5nLgoKVGhlaXIgY291bnRzIHNob3VsZCBiZSB6ZXJvLgoKVGhpcyBvZnRlbiBpbXBhY3RzIHZpc3VhbGl6YXRpb25zLgoKYGNvbXBsZXRlKClgIGNhbiBiZSB1c2VkIHRvIGZpbGwgaW4gdGhlIHplcm8gdmFsdWVzLgoKYGBge3J9CnRibCA8LQogICAgY291bnQoZXEsIFNUQVRFX05BTUUpICU+JQogICAgY29tcGxldGUoU1RBVEVfTkFNRSwgZmlsbCA9IGxpc3QobiA9IDApKQp0YmwKYGBgCgpgYGB7ciwgZmlnLmhlaWdodCA9IDcsIGZpZy53aWR0aCA9IDgsIGNsYXNzLnNvdXJjZSA9ICJmb2xkLWhpZGUifQp0aG1feXR4dF9zbSA8LSB0aGVtZShheGlzLnRleHQueSA9IGVsZW1lbnRfdGV4dChzaXplID0gMTApKQoocGIgJSslIHRibCArIHRobV95dHh0X3NtKSB8IChwcyAlKyUgdGJsICsgdGhtX3l0eHRfc20pCmBgYAoKPCEtLQoKIyBub2xpbnQgc3RhcnQKUGxheSBhcm91bmQgd2l0aCBgdGVlYApgYGB7ciwgZXZhbCA9IEZBTFNFfQp0ZWUgPC0gZnVuY3Rpb24oeCwgZnVuKSB7IGZ1bih4KTsgeCB9CnRlZSguICU+JSBoZWFkKDIpICU+JSBwcmludCkgJT4lCnRlZShicm93c2VyKSAjIyBicm93c2VyVGV4dCBnZXRzIHRoZSBhcmcKIyBub2xpbnQgZW5kCmBgYAoKTWVudGlvbiBgZmlsbGAgYW5kIHN1Y2guCi0tPgoKPCEtLQoKICAgIG9wdGlvbnModGliYmxlLnByaW50X21heCA9IG4sIHRpYmJsZS5wcmludF9taW4gPSBtKTogaWYgdGhlcmUgYXJlCiAgICBtb3JlIHRoYW4gbiByb3dzLCBwcmludCBvbmx5IHRoZSBmaXJzdCBtIHJvd3MuIFVzZQogICAgb3B0aW9ucyh0aWJibGUucHJpbnRfbWF4ID0gSW5mKSB0byBhbHdheXMgc2hvdyBhbGwgcm93cy4KCiAgICBvcHRpb25zKHRpYmJsZS53aWR0aCA9IEluZikgd2lsbCBhbHdheXMgcHJpbnQgYWxsIGNvbHVtbnMsCiAgICByZWdhcmRsZXNzIG9mIHRoZSB3aWR0aCBvZiB0aGUgc2NyZWVuLgoKCiMjICoqKiogdHJ5IHRvIHJlYWQgZnVsbCB0YWJsZSBraWxscyB0aGUgcHJvY2VzcwoKIyMgKioqKiBjaGVjayBpZiBtb3JlIHJlYXNvbmFibGUgb24gUi1kZXZlbAoKLS0+CgoKIyMgUmVhZGluZwoKQ2hhcHRlciBbX0RhdGEgdHJhbnNmb3JtYXRpb25fXShodHRwczovL3I0ZHMuaGFkLmNvLm56L3RyYW5zZm9ybS5odG1sKSBvZgpbUiBmb3IgRGF0YSBTY2llbmNlXShodHRwczovL3I0ZHMuaGFkLmNvLm56LykuCgpDaGFwdGVyIFtfVGlkeSBkYXRhX10oaHR0cHM6Ly9yNGRzLmhhZC5jby5uei90aWR5LWRhdGEuaHRtbCkgb2YKW1IgZm9yIERhdGEgU2NpZW5jZV0oaHR0cHM6Ly9yNGRzLmhhZC5jby5uei8pLgoKCiMjIEludGVyYWN0aXZlIFR1dG9yaWFsCgpBbiBpbnRlcmFjdGl2ZSBbYGxlYXJucmBdKGh0dHBzOi8vcnN0dWRpby5naXRodWIuaW8vbGVhcm5yLykgdHV0b3JpYWwKZm9yIHRoZXNlIG5vdGVzIGlzIFthdmFpbGFibGVdKGByIFdMTksoInR1dG9yaWFscy9kcGx5ci5SbWQiKWApLgoKWW91IGNhbiBydW4gdGhlIHR1dG9yaWFsIHdpdGgKCmBgYHtyLCBldmFsID0gRkFMU0V9ClNUQVQ0NTgwOjpydW5UdXRvcmlhbCgiZHBseXIiKQpgYGAKCllvdSBjYW4gaW5zdGFsbCB0aGUgY3VycmVudCB2ZXJzaW9uIG9mIHRoZSBgU1RBVDQ1ODBgIHBhY2thZ2Ugd2l0aAoKYGBge3IsIGV2YWwgPSBGQUxTRX0KcmVtb3Rlczo6aW5zdGFsbF9naXRsYWIoImx1a2UtdGllcm5leS9TVEFUNDU4MCIpCmBgYAoKWW91IG1heSBuZWVkIHRvIGluc3RhbGwgdGhlIGByZW1vdGVzYCBwYWNrYWdlIGZyb20gQ1JBTiBmaXJzdC4KCgojIyBFeGVyY2lzZXMKCjEuIFRvIGJyaW5nIGluIGBkcGx5cmAgYW5kIHRoZSBgbXBnYCBkYXRhLCBzdGFydCBieSBldmFsdWF0aW5nCgogICAgYGBgcgogICAgbGlicmFyeShkcGx5cikKICAgIGxpYnJhcnkoZ2dwbG90MikKICAgIGBgYAogICAgVGhlIGBzZWxlY3RgIGZ1bmN0aW9uIGFsbG93cyB2YXJpYWJsZXMgdG8gYmUgc3BlY2lmaWVkIGluIGEgdmFyaWV0eSBvZgogICAgd2F5cy4gV2hpY2ggb2YgdGhlIGZvbGxvd2luZyBkb2VzICoqbm90KiogcHJvZHVjZSBhIGRhdGEgZnJhbWUgd2l0aCBvbmx5CiAgICB0aGUgdmFyaWFibGVzIGBtYW51ZmFjdHVyZXJgLCBgbW9kZWxgLCBgY3R5YCwgYGh3eWA/CgogICAgYS4gYHNlbGVjdChtcGcsIDE6MiwgNzo4KWAKICAgIGIuIGBzZWxlY3QobXBnLCBzdGFydHNfd2l0aCgibSIpLCBlbmRzX3dpdGgoInkiKSlgCiAgICBjLiBgc2VsZWN0KG1wZywgMToyLCBjdHkgOiBod3kpYAogICAgZC4gYHNlbGVjdChtcGcsIC0oZGlzcGwgOiBkcnYpLCAtKGZsIDogY2xhc3MpKWAKCjIuIENvbnNpZGVyIHRoZSBjb2RlCgogICAgYGBgcgogICAgbGlicmFyeShkcGx5cikKICAgIGxpYnJhcnkoZ2dwbG90MikKICAgIGZpbHRlcihtcGcsIC0tLSkgJT4lIG5yb3coKQogICAgYGBgCgogICAgV2hpY2ggb2YgdGhlIHJlcGxhY2VtZW50cyBmb3IgYC0tLWAgY29tcHV0ZXMgdGhlIG51bWJlciBvZiBGb3JkCiAgICB2ZWhpY2xlcyB3aXRoIG1vcmUgdGhhbiA2IGN5bGluZGVycyBpbiB0aGUgYG1wZ2AgdGFibGU/CgogICAgYS4gYG1vZGVsID09ICJmb3JkIiwgY3lsIDw9IDZgCiAgICBiLiBgbWFudWZhY3R1cmVyID09ICJmb3JkIiB8IGN5bCA+IDZgCiAgICBjLiBgbWFudWZhY3R1cmVyID09ICJmb3JkIiwgY3lsID4gNmAKICAgIGQuIGBtYW51ZmFjdHVyZXIgPT0gImZvcmQiLCBjeWxpbmRlcnMgPiA2YAoKMy4gSW4gdGhlIDIwMTMgTllDIGZsaWdodHMgZGF0YSBwcm92aWRlZCBieSB0aGUgYG55Y2ZsaWdodHMxM2AgcGFja2FnZQogICBob3cgbWFueSBmbGlnaHRzIHdlcmUgdGhlcmUgdG8gRGVzIE1vaW5lcyAoRkFBIGNvZGUgRFNNKSBmcm9tIE5ZQwogICBpbiB0aGUgZmlyc3QgdGhyZWUgbW9udGhzIG9mIDIwMTM/CgogICAgYS4gMTMKICAgIGIuIDk4CiAgICBjLiA2NAogICAgZC4gNzcKCjQuIFRvIGJyaW5nIGluIGBkcGx5cmAgYW5kIHRoZSBgbXBnYCBkYXRhLCBzdGFydCBieSBldmFsdWF0aW5nCgogICAgYGBgcgogICAgbGlicmFyeShkcGx5cikKICAgIGxpYnJhcnkoZ2dwbG90MikKICAgIGBgYAoKICAgIFdoaWNoIG9mIHRoZSBmb2xsb3dpbmcgc29ydHMgdGhlIHJvd3MgZm9yIGBtcGdgIGJ5IGluY3JlYXNpbmcgYGN5bGAKICAgIHZhbHVlIGFuZCwgd2l0aGluIGVhY2ggYGN5bGAgdmFsdWUgc29ydHMgdGhlIHJvd3MgZnJvbSBsYXJnZXN0IHRvCiAgICBzbWFsbGVzdCBgaHd5YCB2YWx1ZS4KCiAgICBhLiBgYXJyYW5nZShtcGcsIGRlc2MoaHd5KSwgY3lsKWAKICAgIGIuIGBhcnJhbmdlKG1wZywgY3lsLCBkZXNjKGh3eSkpYAogICAgYy4gYGFycmFuZ2UobXBnLCBkZXNjKGN5bCksIGh3eSlgCiAgICBkLiBgYXJyYW5nZShtcGcsIGN5bCwgaHd5KWAKCjUuIFRvIGJyaW5nIGluIGBkcGx5cmAgYW5kIHRoZSBgZmxpZ2h0c2AgZGF0YSwgc3RhcnQgYnkgZXZhbHVhdGluZwoKICAgIGBgYHIKICAgIGxpYnJhcnkoZHBseXIpCiAgICBsaWJyYXJ5KG55Y2ZsaWdodHMxMykKICAgIGBgYAoKICAgIFRoZSBgZGVwX3RpbWVgIHZhcmlhYmxlIGluIHRoZSBgZmxpZ2h0c2AgZGF0YSBzZXQgZnJvbSB0aGUKICAgIGBueWNmbGlnaHRzMTNgIHBhY2thZ2UgaXMgY29udmVuaWVudCB0byBsb29rIGF0ICg1MjkgbWVhbnMgNToyOSBBTQogICAgYW5kIDIyOjEyIG1lYW5zIDEwOjEyIFBNKSwgYnV0IGhhcmQgdG8gY29tcHV0ZSB3aXRoLiBXaGljaCBvZiB0aGUKICAgIGZvbGxvd2luZyBhZGRzIHZhcmlhYmxlcyBgZGVwX2hvdXJgIGFuZCBgZGVwX21pbmAgY29udGFpbmluZyBob3VyCiAgICBhbmQgbWludXRlIG9mIHRoZSBkZXBhcnR1cmUgdGltZT8KCiAgICBhLiBgbXV0YXRlKGZsaWdodHMsIGRlcF9ob3VyID0gZGVwX3RpbWUgJS8lIDYwLCBkZXBfbWluID0gZGVwX3RpbWUgJSUgNjApYAogICAgYi4gYG11dGF0ZShmbGlnaHRzLCBkZXBfaG91ciA9IGRlcF90aW1lICUvJSAxMDAsIGRlcF9taW4gPSBkZXBfdGltZSAlJSAxMDApYAogICAgYy4gYG11dGF0ZShmbGlnaHRzLCBkZXBfaG91ciA9IGRlcF90aW1lIC8gMTAwLCBkZXBfbWluID0gZGVwX3RpbWUgLSBkZXBfaG91cilgCiAgICBhLiBgbXV0YXRlKGZsaWdodHMsIGRlcF9ob3VyID0gZGVwX3RpbWUgJS8lIDYwLCBkZXBfbWluID0gZGVwX2hvdXIgJSUgNjApYAoKNi4gVXNpbmcgdGhlIGBnYXBtaW5kZXJgIGRhdGEgc2V0LCB0aGUgZm9sbG93aW5nIGNvZGUgY29tcHV0ZXMKICAgIHBvcHVsYXRpb24gdG90YWxzIGZvciBlYWNoIGNvbnRpbmVudCBhbmQgZWFjaCB5ZWFyOgoKICAgIGBgYHtyLCBtZXNzYWdlID0gRkFMU0V9CiAgICBsaWJyYXJ5KGRwbHlyKQogICAgbGlicmFyeShnYXBtaW5kZXIpCiAgICBjcG9wcyA8LSBncm91cF9ieShnYXBtaW5kZXIsIGNvbnRpbmVudCwgeWVhcikgJT4lCiAgICAgICAgc3VtbWFyaXplKHBvcCA9IHN1bShwb3ApKSAlPiUKICAgICAgICB1bmdyb3VwKCkKICAgIGBgYAoKICAgIFRvIHByb2R1Y2UgYSBwbG90IHRvIGNvbXBhcmUgcG9wdWxhdGlvbiBncm93dGggb3ZlciB0aGUgeWVhcnMgZm9yCiAgICB0aGUgY29udGluZW50cyBpdCBpcyB1c2VmdWwgdG8gc3RhbmRhcmRpemUgdGhlIHBvcHVsYXRpb24gZGF0YSBmb3IKICAgIGVhY2ggY29udGluZW50LCBmb3IgZXhhbXBsZSBieSBkaXZpZGluZyB0aGUgcG9wdWxhdGlvbiB2YWx1ZXMgYnkKICAgIHRoZSBhdmVyYWdlIHBvcHVsYXRpb24gc2l6ZSBmb3IgZWFjaCBjb250aW5lbnQuIE9uZSB3YXkgdG8gZG8gdGhpcwogICAgaXMgd2l0aCBhIGdyb3VwZWQgYG11dGF0ZWAuIFRoZSBmaXJzdCBsaW5lIG9mIHlvdXIgcmVzdWx0IHNob3VsZCBiZQoKICAgIGBgYHtyLCBpbmNsdWRlID0gRkFMU0V9CiAgICBjcG9wc19zdGQgPC0gZ3JvdXBfYnkoY3BvcHMsIGNvbnRpbmVudCkgJT4lCiAgICAgICAgbXV0YXRlKHN0ZHBvcCA9IHBvcCAvIG1lYW4ocG9wKSkgJT4lCiAgICAgICAgdW5ncm91cCgpCiAgICBgYGAKICAgIGBgYHtyfQogICAgaGVhZChjcG9wc19zdGQsIDEpCiAgICBgYGAKCiAgICBgYGB7ciwgZmlnLmNhcCA9ICIifQogICAgbGlicmFyeShnZ3Bsb3QyKQogICAgZ2dwbG90KGNwb3BzX3N0ZCwgYWVzKHggPSB5ZWFyLCB5ID0gc3RkcG9wLCBjb2xvciA9IGNvbnRpbmVudCkpICsKICAgICAgICBnZW9tX2xpbmUoKQogICAgYGBgCgogICAgV2hpY2ggb2YgdGhlIGZvbGxvd2luZyBwcm9kdWNlcyB0aGUgY29ycmVjdCByZXN1bHQ6CgogICAgYS4gYGNwb3BzX3N0ZCA8LSBncm91cF9ieShjcG9wcywgY29udGluZW50KSAlPiUgbXV0YXRlKHN0ZHBvcCA9IHBvcCAvIG1lYW4oeWVhcikpICU+JSB1bmdyb3VwKClgCiAgICBhLiBgY3BvcHNfc3RkIDwtIGdyb3VwX2J5KGNwb3BzLCB5ZWFyKSAlPiUgbXV0YXRlKHN0ZHBvcCA9IHBvcCAvIG1lYW4ocG9wKSkgJT4lIHVuZ3JvdXAoKWAKICAgIGMuIGBjcG9wc19zdGQgPC0gZ3JvdXBfYnkoY3BvcHMsIGNvbnRpbmVudCkgJT4lIG11dGF0ZShzdGRwb3AgPSBwb3AgLyBtZWFuKHBvcCkpICU+JSB1bmdyb3VwKClgCiAgICBkLiBgY3BvcHNfc3RkIDwtIGdyb3VwX2J5KGNwb3BzLCB5ZWFyKSAlPiUgbXV0YXRlKHN0ZHBvcCA9IHBvcCAvIG1lYW4oeWVhcikpICU+JSB1bmdyb3VwKClgCgo3LiBBbm90aGVyIGFwcHJvYWNoIHRvIHRoZSBwcmV2aW91cyBleGVyY2lzZSBmaXJzdCBjcmVhdGVzIGEgdGFibGUgb2YKICAgcG9wdWxhdGlvbiBhdmVyYWdlcyB3aXRoCgogICAgYGBgcgogICAgY3BvcHNfYXZnIDwtIGdyb3VwX2J5KGNwb3BzLCBjb250aW5lbnQpICU+JQogICAgICAgIHN1bW1hcml6ZShhdmdfcG9wID0gbWVhbihwb3ApKQogICAgYGBgCgogICAgVGhlbiB1c2UgYSBsZWZ0IGpvaW4gdG8gYWRkIGBhdmdfcG9wYCB0byB0aGUgYGNwb3BzYCB0YWJsZSwKICAgIGZvbGxvd2VkIGJ5IGFuIG9yZGluYXJ5IG11dGF0ZSBzdGVwOgoKICAgIGBgYHIKICAgIGxlZnRfam9pbigtLS0pICU+JQogICAgICAgIG11dGF0ZShzdGRwb3AgPSBwb3AgLyBhdmdfcG9wKQogICAgYGBgCgogICAgV2hpY2ggaXMgdGhlIGNvcnJlY3QgcmVwbGFjZW1lbnQgZm9yIGAtLS1gPwoKICAgIGEuIGBjcG9wc19hdmcsIGNwb3BzLCAiY29udGluZW50ImAKICAgIGIuIGBjcG9wcywgY3BvcHNfYXZnLCAiY29udGluZW50ImAKICAgIGMuIGBjcG9wcywgY3BvcHNfYXZnLCAieWVhciJgCiAgICBkLiBgY3BvcHNfYXZnLCBjcG9wcywgInllYXIiYAo=