Solutions to this workshop can be found here

Testing functions

One really critical point is that you really need test any new functions extensively. It is incredibly easy to make a mistake in coding (think about all the typos you’ve ever made, except this time they can critically affect the results of your data analysis). The solution to this is testing any code you write extensively, and if possible, having another coder familiar with what you’re doing look over it (which means always including tons of comments!)

Before we do anything else, we’ll need to load in the function we created last time to grow plants, since we’re going to be re-using this. To do this, run the source() command, with the full path to grow_plants.R as the argument

# something like this
source('~/Documents/Teaching/Intro_R_Course/grow_plants.R')

Now, let’s run some tests.

# set the variables you'll use for plant growth
# (sample_size, plant_growth_per_day, plant_growth_sd)

# re-initialize plant_growth_df at day 0, as yesterday

# Test your function on plant_growth_df, adding a day of growth and
# checking that the results look like you'd expect
# First, test what happens if you set the mean and standard deviation of the
# daily growth amount to 0

# Next, set the mean to a non-0 number, but the standard deviation to 0

# Finally, try growing the plants with a non-0 mean and standard deviation
# (e.g. plant_growth_per_day and plant_growth_sd from above)

Another great sanity check in coding anything related to data is plotting. Try making a plot of your data, plotting height vs day for all the plants.

# Plot plant_growth_df

# If you want to include lines on your plot, you will have to include a "group"
# parameter in your mapping (i.e. "group = plant.id"); this separates the data
# by the category (or categories) you specify, without giving it a special color,
# shape, or associated legend

Loops

Introduction to loops

We now have a function that allows us to ‘grow’ the plants in our dataframe in a single line, just by providing the growth parameters. However, this doesn’t actually solve our problem of trying to model the growth of these plants over long periods of time. We ideally want to avoid running this model by hand 30+ times.

Instead, we can use something called a “loop”. These are an incredibly fundamental idea in programming, and they allow you to easily repeat a task multiple times.

for loops

The most commonly used type of loop is a ‘for’ loop. Imagine we have some list (e.g. a consecutive list of days). The loop takes a variable and takes turns assigning every value from the list to it, in order, and then doing something with the variable. Here’s how they look in R:

for (variable_from_list in some_list){
  # do some stuff using variable_from_list
}

Note that unlike functions, loops are not closed-off boxes: in addition to variable_from_list, they have access to all the previously specified variables in R, and they can modify these variables.

Let’s try an example: we can write a loop that starts with a vector of numbers, and produces an output vector of those numbers squared.

# numbers to loop through
initial_vector <- c(0, 3, -2, 4, 15)

# need to initialize an empty vector into which we will put the solutions in
# each loop round
squares_vector <- c()

# loop over initial_vector, and populate square_vector
for(current_number in initial_vector){
  # print current_number, just so we can see what values it takes on
  print(current_number)
  # notice that current_number will take turns holding every value of initial_vector
  current_number_squared <- current_number^2
  # add current_number_squared to the end of squares_vector
  squares_vector <- c(squares_vector, current_number_squared)
}
print(squares_vector)

Of course, you don’t need a loop to do the above; simply running initial_vector^2 would have worked. In fact, running this kind of operation in the loop we wrote takes much longer than the alternative. However, there are many cases (like our plant growth problem) where loops are the only way to do something.

while loops

There’s also another kind of loop, a ‘while’ loop. Rather than going through a vector of pre-defined length, while loops check that some condition is true, and only continue while that is the case.

Here’s a while loop that subtracts 2 from a number and prints the result, stopping only when the result is equal to 0.

# initialize the while loop
number <- 8
while (number != 0){
  number <- number - 2
  print(number)
}

while() loops are tricky, however. If you don’t think carefully about your starting condition, you can end up in an endless loop. For example, try changing the initial value of number to 7 in the loop above.

Hint: You can press inside your console and hold down ctrl-c or Esc to end the execution of any R code.

Coding plant growth in a loop

Now that we know how to write loops, try to write some code that makes a new dataframe, plant_growth_df_2, and then uses a loop running the grow_plants function to loop through 30 days of plant growth.

# Initialize plant_growth_df_2 for day 0

# Loop through days 1-30, adding to plant growth

# plot plant growth over time

Putting it all together: Using simulations to plan experiments

Let’s come back to our initial problem. We want to figure out whether salt affects plant growth. To do this, we plan to compare two conditions: plants grown without salt, and plants grown with some standard concentration of salt added to the soil.

Imagine we suspect, based on previous literature, that salt decreases plant growth rate by 20%. How large of an experiment would we have to do to see this change? How long do we need to run this experiment for?

We can try to get ballpark answers to these questions with simulations.

First, let’s convert our plant dataframe-creating loop from above into a function. In addition to all the plant growth parameters, we may want to pass the number of individual plants we’re growing, as well as the duration of growth, into this function.

# Write a function, simulate_growth, that takes in mean and s.d. of daily plant
# growth, the number of plants to simulate, and the number of days to 'grow'
# the plants for
# Output should be a dataframe of plant heights over time

Now we can use our new simulate_growth() function to simulate plant growth with and without salt.

# Create variables for mean daily growth amoung with and without salt, s.d. of
# daily growth, # of plants to simulate, and number of days to grow the plants

# Use the variables above to create two dataframes: growth_with_salt, and
# growth_no_salt

# Add a 'Condition' column to each of the new dataframes that will say whether
# or not salt was used

# Use rbind to combine the two dataframes into a new dataframe, combined_plant_df

Now that we’ve simulated our data, we can do some plotting and statistical analysis on it!

# Plot the timecourse data from combined_plant_df in a useful way
# (coloring by Condition, with an individual line for each plant)

# This may not have worked as you expected because as far as ggplot is concerned,
# the individuals labeled "1" are all the same individual, regardless of whether
# they come from the salt condition or not. We have to tell ggplot this is not
# the case
combined_plant_df$plant.id.full <-
  paste(combined_plant_df$Condition, combined_plant_df$plant.id)
View(combined_plant_df)
# This is a *very* common problem with data that comes from multiple experiments
# (e.g. well location is unique within a 96-well plate, but you run into
# problems when combining data from multiple plates)

# repeat the plot, using the plant.id.full column

# Use lm to model the effect of Condition on plant growth, and check the output
# Is the observed change significant?

What would you do if you wanted to repeat this simulation and comparison 1000 times, saving the p-value associated with the effect of Condition each time?

Using coding in science

I wanted to share some final resources that I think are great places to look to keep going with learning to code.

If you’re planning to spend a lot of time coding (i.e. not just a one-off analysis, but more long-term projects), the following articles are great sources for thinking about what to aspire to. They can be daunting if you try to achieve all their suggestions at once, but I think as they both try to point out, a better idea is understanding what good, reproducible coding looks like and then slowly building more and more of these practices into your work as you grow. Few coding projects in biology manage to achieve everything outlined here, but these articles provide an excellent long-term roadmap as you become a better coder.

LS0tCnRpdGxlOiAiSW50cm8gUiBDb3Vyc2UsIFdvcmtzaG9wIDk6IEJhc2ljcyBvZiBwcm9ncmFtbWluZywgYW5kIHNpbXVsYXRpb24gaW4gYmlvbG9neSAoUGFydCBJSSkiCnN1YnRpdGxlOiB8CiAgICB8ICAgLSBzaW11bGF0aW5nIGJpb2xvZ2ljYWwgZGF0YSAoY29udGludWVkKQogICAgfCAgIC0gdGVzdGluZyBmdW5jdGlvbnMKICAgIHwgICAtIGxvb3BzCiAgICB8ICAgLSB1c2luZyBzaW11bGF0aW9ucyB0byBwbGFuIGV4cGVyaW1lbnRzCiAgICB8ICAgLSBhZGRpdGlvbmFsIHJlc291cmNlcwphdXRob3I6CiAgLSBFdWdlbmUgUGxhdnNraW4Kb3V0cHV0OgogIGh0bWxfbm90ZWJvb2s6CiAgICBjb2RlX2ZvbGRpbmc6IHNob3cKICAgIGRlcHRoOiAzCiAgICB0aWR5OiB5ZXMKICAgIHRvYzogeWVzCi0tLQoqKlNvbHV0aW9ucyB0byB0aGlzIHdvcmtzaG9wIGNhbiBiZSBmb3VuZCBbaGVyZV0oU29sdXRpb25zX1dvcmtzaG9wXzkubmIuaHRtbCkqKgoKIyBUZXN0aW5nIGZ1bmN0aW9ucwoKT25lIHJlYWxseSBjcml0aWNhbCBwb2ludCBpcyB0aGF0IHlvdSByZWFsbHkgbmVlZCB0ZXN0IGFueSBuZXcgZnVuY3Rpb25zIGV4dGVuc2l2ZWx5LiBJdCBpcyAqKmluY3JlZGlibHkqKiBlYXN5IHRvIG1ha2UgYSBtaXN0YWtlIGluIGNvZGluZyAodGhpbmsgYWJvdXQgYWxsIHRoZSB0eXBvcyB5b3UndmUgZXZlciBtYWRlLCBleGNlcHQgdGhpcyB0aW1lIHRoZXkgY2FuIGNyaXRpY2FsbHkgYWZmZWN0IHRoZSByZXN1bHRzIG9mIHlvdXIgZGF0YSBhbmFseXNpcykuIFRoZSBzb2x1dGlvbiB0byB0aGlzIGlzIHRlc3RpbmcgYW55IGNvZGUgeW91IHdyaXRlIGV4dGVuc2l2ZWx5LCBhbmQgaWYgcG9zc2libGUsIGhhdmluZyBhbm90aGVyIGNvZGVyIGZhbWlsaWFyIHdpdGggd2hhdCB5b3UncmUgZG9pbmcgbG9vayBvdmVyIGl0ICh3aGljaCBtZWFucyBhbHdheXMgaW5jbHVkaW5nIHRvbnMgb2YgY29tbWVudHMhKQoKQmVmb3JlIHdlIGRvIGFueXRoaW5nIGVsc2UsIHdlJ2xsIG5lZWQgdG8gbG9hZCBpbiB0aGUgZnVuY3Rpb24gd2UgY3JlYXRlZCBsYXN0IHRpbWUgdG8gZ3JvdyBwbGFudHMsIHNpbmNlIHdlJ3JlIGdvaW5nIHRvIGJlIHJlLXVzaW5nIHRoaXMuIFRvIGRvIHRoaXMsIHJ1biB0aGUgYHNvdXJjZSgpYCBjb21tYW5kLCB3aXRoIHRoZSBmdWxsIHBhdGggdG8gKmdyb3dfcGxhbnRzLlIqIGFzIHRoZSBhcmd1bWVudApgYGB7cn0KIyBzb21ldGhpbmcgbGlrZSB0aGlzCnNvdXJjZSgnfi9Eb2N1bWVudHMvVGVhY2hpbmcvSW50cm9fUl9Db3Vyc2UvZ3Jvd19wbGFudHMuUicpCmBgYAoKTm93LCBsZXQncyBydW4gc29tZSB0ZXN0cy4KYGBge3J9CiMgc2V0IHRoZSB2YXJpYWJsZXMgeW91J2xsIHVzZSBmb3IgcGxhbnQgZ3Jvd3RoCiMgKHNhbXBsZV9zaXplLCBwbGFudF9ncm93dGhfcGVyX2RheSwgcGxhbnRfZ3Jvd3RoX3NkKQoKIyByZS1pbml0aWFsaXplIHBsYW50X2dyb3d0aF9kZiBhdCBkYXkgMCwgYXMgeWVzdGVyZGF5CgojIFRlc3QgeW91ciBmdW5jdGlvbiBvbiBwbGFudF9ncm93dGhfZGYsIGFkZGluZyBhIGRheSBvZiBncm93dGggYW5kCiMgY2hlY2tpbmcgdGhhdCB0aGUgcmVzdWx0cyBsb29rIGxpa2UgeW91J2QgZXhwZWN0CiMgRmlyc3QsIHRlc3Qgd2hhdCBoYXBwZW5zIGlmIHlvdSBzZXQgdGhlIG1lYW4gYW5kIHN0YW5kYXJkIGRldmlhdGlvbiBvZiB0aGUKIyBkYWlseSBncm93dGggYW1vdW50IHRvIDAKCiMgTmV4dCwgc2V0IHRoZSBtZWFuIHRvIGEgbm9uLTAgbnVtYmVyLCBidXQgdGhlIHN0YW5kYXJkIGRldmlhdGlvbiB0byAwCgojIEZpbmFsbHksIHRyeSBncm93aW5nIHRoZSBwbGFudHMgd2l0aCBhIG5vbi0wIG1lYW4gYW5kIHN0YW5kYXJkIGRldmlhdGlvbgojIChlLmcuIHBsYW50X2dyb3d0aF9wZXJfZGF5IGFuZCBwbGFudF9ncm93dGhfc2QgZnJvbSBhYm92ZSkKCmBgYAoKQW5vdGhlciBncmVhdCBzYW5pdHkgY2hlY2sgaW4gY29kaW5nIGFueXRoaW5nIHJlbGF0ZWQgdG8gZGF0YSBpcyBwbG90dGluZy4gVHJ5IG1ha2luZyBhIHBsb3Qgb2YgeW91ciBkYXRhLCBwbG90dGluZyBoZWlnaHQgdnMgZGF5IGZvciBhbGwgdGhlIHBsYW50cy4KYGBge3J9CiMgUGxvdCBwbGFudF9ncm93dGhfZGYKCiMgSWYgeW91IHdhbnQgdG8gaW5jbHVkZSBsaW5lcyBvbiB5b3VyIHBsb3QsIHlvdSB3aWxsIGhhdmUgdG8gaW5jbHVkZSBhICJncm91cCIKIyBwYXJhbWV0ZXIgaW4geW91ciBtYXBwaW5nIChpLmUuICJncm91cCA9IHBsYW50LmlkIik7IHRoaXMgc2VwYXJhdGVzIHRoZSBkYXRhCiMgYnkgdGhlIGNhdGVnb3J5IChvciBjYXRlZ29yaWVzKSB5b3Ugc3BlY2lmeSwgd2l0aG91dCBnaXZpbmcgaXQgYSBzcGVjaWFsIGNvbG9yLAojIHNoYXBlLCBvciBhc3NvY2lhdGVkIGxlZ2VuZAoKYGBgCgojIExvb3BzCgojIyBJbnRyb2R1Y3Rpb24gdG8gbG9vcHMKCldlIG5vdyBoYXZlIGEgZnVuY3Rpb24gdGhhdCBhbGxvd3MgdXMgdG8gJ2dyb3cnIHRoZSBwbGFudHMgaW4gb3VyIGRhdGFmcmFtZSBpbiBhIHNpbmdsZSBsaW5lLCBqdXN0IGJ5IHByb3ZpZGluZyB0aGUgZ3Jvd3RoIHBhcmFtZXRlcnMuIEhvd2V2ZXIsIHRoaXMgZG9lc24ndCBhY3R1YWxseSBzb2x2ZSBvdXIgcHJvYmxlbSBvZiB0cnlpbmcgdG8gbW9kZWwgdGhlIGdyb3d0aCBvZiB0aGVzZSBwbGFudHMgb3ZlciBsb25nIHBlcmlvZHMgb2YgdGltZS4gV2UgaWRlYWxseSB3YW50IHRvIGF2b2lkIHJ1bm5pbmcgdGhpcyBtb2RlbCBieSBoYW5kIDMwKyB0aW1lcy4KCkluc3RlYWQsIHdlIGNhbiB1c2Ugc29tZXRoaW5nIGNhbGxlZCBhICJsb29wIi4gVGhlc2UgYXJlIGFuIGluY3JlZGlibHkgZnVuZGFtZW50YWwgaWRlYSBpbiBwcm9ncmFtbWluZywgYW5kIHRoZXkgYWxsb3cgeW91IHRvIGVhc2lseSByZXBlYXQgYSB0YXNrIG11bHRpcGxlIHRpbWVzLgoKIyMjIGBmb3JgIGxvb3BzClRoZSBtb3N0IGNvbW1vbmx5IHVzZWQgdHlwZSBvZiBsb29wIGlzIGEgJ2ZvcicgbG9vcC4gSW1hZ2luZSB3ZSBoYXZlIHNvbWUgbGlzdCAoZS5nLiBhIGNvbnNlY3V0aXZlIGxpc3Qgb2YgZGF5cykuIFRoZSBsb29wIHRha2VzIGEgdmFyaWFibGUgYW5kIHRha2VzIHR1cm5zIGFzc2lnbmluZyBldmVyeSB2YWx1ZSBmcm9tIHRoZSBsaXN0IHRvIGl0LCBpbiBvcmRlciwgYW5kIHRoZW4gZG9pbmcgc29tZXRoaW5nIHdpdGggdGhlIHZhcmlhYmxlLiBIZXJlJ3MgaG93IHRoZXkgbG9vayBpbiBSOgpgYGB7fQpmb3IgKHZhcmlhYmxlX2Zyb21fbGlzdCBpbiBzb21lX2xpc3QpewogICMgZG8gc29tZSBzdHVmZiB1c2luZyB2YXJpYWJsZV9mcm9tX2xpc3QKfQpgYGAKTm90ZSB0aGF0IHVubGlrZSBmdW5jdGlvbnMsIGxvb3BzIGFyZSAqbm90KiBjbG9zZWQtb2ZmIGJveGVzOiBpbiBhZGRpdGlvbiB0byB2YXJpYWJsZV9mcm9tX2xpc3QsIHRoZXkgaGF2ZSBhY2Nlc3MgdG8gYWxsIHRoZSBwcmV2aW91c2x5IHNwZWNpZmllZCB2YXJpYWJsZXMgaW4gUiwgYW5kIHRoZXkgY2FuIG1vZGlmeSB0aGVzZSB2YXJpYWJsZXMuCgpMZXQncyB0cnkgYW4gZXhhbXBsZTogd2UgY2FuIHdyaXRlIGEgbG9vcCB0aGF0IHN0YXJ0cyB3aXRoIGEgdmVjdG9yIG9mIG51bWJlcnMsIGFuZCBwcm9kdWNlcyBhbiBvdXRwdXQgdmVjdG9yIG9mIHRob3NlIG51bWJlcnMgc3F1YXJlZC4KYGBge3J9CiMgbnVtYmVycyB0byBsb29wIHRocm91Z2gKaW5pdGlhbF92ZWN0b3IgPC0gYygwLCAzLCAtMiwgNCwgMTUpCgojIG5lZWQgdG8gaW5pdGlhbGl6ZSBhbiBlbXB0eSB2ZWN0b3IgaW50byB3aGljaCB3ZSB3aWxsIHB1dCB0aGUgc29sdXRpb25zIGluCiMgZWFjaCBsb29wIHJvdW5kCnNxdWFyZXNfdmVjdG9yIDwtIGMoKQoKIyBsb29wIG92ZXIgaW5pdGlhbF92ZWN0b3IsIGFuZCBwb3B1bGF0ZSBzcXVhcmVfdmVjdG9yCmZvcihjdXJyZW50X251bWJlciBpbiBpbml0aWFsX3ZlY3Rvcil7CiAgIyBwcmludCBjdXJyZW50X251bWJlciwganVzdCBzbyB3ZSBjYW4gc2VlIHdoYXQgdmFsdWVzIGl0IHRha2VzIG9uCiAgcHJpbnQoY3VycmVudF9udW1iZXIpCiAgIyBub3RpY2UgdGhhdCBjdXJyZW50X251bWJlciB3aWxsIHRha2UgdHVybnMgaG9sZGluZyBldmVyeSB2YWx1ZSBvZiBpbml0aWFsX3ZlY3RvcgogIGN1cnJlbnRfbnVtYmVyX3NxdWFyZWQgPC0gY3VycmVudF9udW1iZXJeMgogICMgYWRkIGN1cnJlbnRfbnVtYmVyX3NxdWFyZWQgdG8gdGhlIGVuZCBvZiBzcXVhcmVzX3ZlY3RvcgogIHNxdWFyZXNfdmVjdG9yIDwtIGMoc3F1YXJlc192ZWN0b3IsIGN1cnJlbnRfbnVtYmVyX3NxdWFyZWQpCn0KcHJpbnQoc3F1YXJlc192ZWN0b3IpCmBgYApPZiBjb3Vyc2UsIHlvdSBkb24ndCBuZWVkIGEgbG9vcCB0byBkbyB0aGUgYWJvdmU7IHNpbXBseSBydW5uaW5nIGBpbml0aWFsX3ZlY3Rvcl4yYCB3b3VsZCBoYXZlIHdvcmtlZC4gSW4gZmFjdCwgcnVubmluZyB0aGlzIGtpbmQgb2Ygb3BlcmF0aW9uIGluIHRoZSBsb29wIHdlIHdyb3RlIHRha2VzICptdWNoKiBsb25nZXIgdGhhbiB0aGUgYWx0ZXJuYXRpdmUuIEhvd2V2ZXIsIHRoZXJlIGFyZSBtYW55IGNhc2VzIChsaWtlIG91ciBwbGFudCBncm93dGggcHJvYmxlbSkgd2hlcmUgbG9vcHMgYXJlIHRoZSBvbmx5IHdheSB0byBkbyBzb21ldGhpbmcuCgojIyMgYHdoaWxlYCBsb29wcwoKVGhlcmUncyBhbHNvIGFub3RoZXIga2luZCBvZiBsb29wLCBhICd3aGlsZScgbG9vcC4gUmF0aGVyIHRoYW4gZ29pbmcgdGhyb3VnaCBhIHZlY3RvciBvZiBwcmUtZGVmaW5lZCBsZW5ndGgsIHdoaWxlIGxvb3BzIGNoZWNrIHRoYXQgc29tZSBjb25kaXRpb24gaXMgdHJ1ZSwgYW5kIG9ubHkgY29udGludWUgKndoaWxlKiB0aGF0IGlzIHRoZSBjYXNlLgoKSGVyZSdzIGEgd2hpbGUgbG9vcCB0aGF0IHN1YnRyYWN0cyAyIGZyb20gYSBudW1iZXIgYW5kIHByaW50cyB0aGUgcmVzdWx0LCBzdG9wcGluZyBvbmx5IHdoZW4gdGhlIHJlc3VsdCBpcyBlcXVhbCB0byAwLgpgYGB7cn0KIyBpbml0aWFsaXplIHRoZSB3aGlsZSBsb29wCm51bWJlciA8LSA4CndoaWxlIChudW1iZXIgIT0gMCl7CiAgbnVtYmVyIDwtIG51bWJlciAtIDIKICBwcmludChudW1iZXIpCn0KYGBgCmB3aGlsZSgpYCBsb29wcyBhcmUgdHJpY2t5LCBob3dldmVyLiBJZiB5b3UgZG9uJ3QgdGhpbmsgY2FyZWZ1bGx5IGFib3V0IHlvdXIgc3RhcnRpbmcgY29uZGl0aW9uLCB5b3UgY2FuIGVuZCB1cCBpbiBhbiBlbmRsZXNzIGxvb3AuIEZvciBleGFtcGxlLCB0cnkgY2hhbmdpbmcgdGhlIGluaXRpYWwgdmFsdWUgb2YgYG51bWJlcmAgdG8gNyBpbiB0aGUgbG9vcCBhYm92ZS4KCioqSGludCoqOiBZb3UgY2FuIHByZXNzIGluc2lkZSB5b3VyIGNvbnNvbGUgYW5kIGhvbGQgZG93biBgY3RybC1jYCBvciBgRXNjYCB0byBlbmQgdGhlIGV4ZWN1dGlvbiBvZiBhbnkgUiBjb2RlLgoKIyMgQ29kaW5nIHBsYW50IGdyb3d0aCBpbiBhIGxvb3AKCk5vdyB0aGF0IHdlIGtub3cgaG93IHRvIHdyaXRlIGxvb3BzLCB0cnkgdG8gd3JpdGUgc29tZSBjb2RlIHRoYXQgbWFrZXMgYSBuZXcgZGF0YWZyYW1lLCBwbGFudF9ncm93dGhfZGZfMiwgYW5kIHRoZW4gdXNlcyBhIGxvb3AgcnVubmluZyB0aGUgZ3Jvd19wbGFudHMgZnVuY3Rpb24gdG8gbG9vcCB0aHJvdWdoIDMwIGRheXMgb2YgcGxhbnQgZ3Jvd3RoLgpgYGB7cn0KIyBJbml0aWFsaXplIHBsYW50X2dyb3d0aF9kZl8yIGZvciBkYXkgMAoKIyBMb29wIHRocm91Z2ggZGF5cyAxLTMwLCBhZGRpbmcgdG8gcGxhbnQgZ3Jvd3RoCgojIHBsb3QgcGxhbnQgZ3Jvd3RoIG92ZXIgdGltZQoKYGBgCgojIFB1dHRpbmcgaXQgYWxsIHRvZ2V0aGVyOiBVc2luZyBzaW11bGF0aW9ucyB0byBwbGFuIGV4cGVyaW1lbnRzCgpMZXQncyBjb21lIGJhY2sgdG8gb3VyIGluaXRpYWwgcHJvYmxlbS4gV2Ugd2FudCB0byBmaWd1cmUgb3V0IHdoZXRoZXIgc2FsdCBhZmZlY3RzIHBsYW50IGdyb3d0aC4gVG8gZG8gdGhpcywgd2UgcGxhbiB0byBjb21wYXJlIHR3byBjb25kaXRpb25zOiBwbGFudHMgZ3Jvd24gd2l0aG91dCBzYWx0LCBhbmQgcGxhbnRzIGdyb3duIHdpdGggc29tZSBzdGFuZGFyZCBjb25jZW50cmF0aW9uIG9mIHNhbHQgYWRkZWQgdG8gdGhlIHNvaWwuCgpJbWFnaW5lIHdlIHN1c3BlY3QsIGJhc2VkIG9uIHByZXZpb3VzIGxpdGVyYXR1cmUsIHRoYXQgc2FsdCBkZWNyZWFzZXMgcGxhbnQgZ3Jvd3RoIHJhdGUgYnkgMjAlLiBIb3cgbGFyZ2Ugb2YgYW4gZXhwZXJpbWVudCB3b3VsZCB3ZSBoYXZlIHRvIGRvIHRvIHNlZSB0aGlzIGNoYW5nZT8gSG93IGxvbmcgZG8gd2UgbmVlZCB0byBydW4gdGhpcyBleHBlcmltZW50IGZvcj8KCldlIGNhbiB0cnkgdG8gZ2V0IGJhbGxwYXJrIGFuc3dlcnMgdG8gdGhlc2UgcXVlc3Rpb25zIHdpdGggc2ltdWxhdGlvbnMuCgpGaXJzdCwgbGV0J3MgY29udmVydCBvdXIgcGxhbnQgZGF0YWZyYW1lLWNyZWF0aW5nIGxvb3AgZnJvbSBhYm92ZSBpbnRvIGEgZnVuY3Rpb24uIEluIGFkZGl0aW9uIHRvIGFsbCB0aGUgcGxhbnQgZ3Jvd3RoIHBhcmFtZXRlcnMsIHdlIG1heSB3YW50IHRvIHBhc3MgdGhlIG51bWJlciBvZiBpbmRpdmlkdWFsIHBsYW50cyB3ZSdyZSBncm93aW5nLCBhcyB3ZWxsIGFzIHRoZSBkdXJhdGlvbiBvZiBncm93dGgsIGludG8gdGhpcyBmdW5jdGlvbi4KYGBge3J9CiMgV3JpdGUgYSBmdW5jdGlvbiwgc2ltdWxhdGVfZ3Jvd3RoLCB0aGF0IHRha2VzIGluIG1lYW4gYW5kIHMuZC4gb2YgZGFpbHkgcGxhbnQKIyBncm93dGgsIHRoZSBudW1iZXIgb2YgcGxhbnRzIHRvIHNpbXVsYXRlLCBhbmQgdGhlIG51bWJlciBvZiBkYXlzIHRvICdncm93JwojIHRoZSBwbGFudHMgZm9yCiMgT3V0cHV0IHNob3VsZCBiZSBhIGRhdGFmcmFtZSBvZiBwbGFudCBoZWlnaHRzIG92ZXIgdGltZQoKYGBgCgpOb3cgd2UgY2FuIHVzZSBvdXIgbmV3IGBzaW11bGF0ZV9ncm93dGgoKWAgZnVuY3Rpb24gdG8gc2ltdWxhdGUgcGxhbnQgZ3Jvd3RoIHdpdGggYW5kIHdpdGhvdXQgc2FsdC4KYGBge3J9CiMgQ3JlYXRlIHZhcmlhYmxlcyBmb3IgbWVhbiBkYWlseSBncm93dGggYW1vdW5nIHdpdGggYW5kIHdpdGhvdXQgc2FsdCwgcy5kLiBvZgojIGRhaWx5IGdyb3d0aCwgIyBvZiBwbGFudHMgdG8gc2ltdWxhdGUsIGFuZCBudW1iZXIgb2YgZGF5cyB0byBncm93IHRoZSBwbGFudHMKCiMgVXNlIHRoZSB2YXJpYWJsZXMgYWJvdmUgdG8gY3JlYXRlIHR3byBkYXRhZnJhbWVzOiBncm93dGhfd2l0aF9zYWx0LCBhbmQKIyBncm93dGhfbm9fc2FsdAoKIyBBZGQgYSAnQ29uZGl0aW9uJyBjb2x1bW4gdG8gZWFjaCBvZiB0aGUgbmV3IGRhdGFmcmFtZXMgdGhhdCB3aWxsIHNheSB3aGV0aGVyCiMgb3Igbm90IHNhbHQgd2FzIHVzZWQKCiMgVXNlIHJiaW5kIHRvIGNvbWJpbmUgdGhlIHR3byBkYXRhZnJhbWVzIGludG8gYSBuZXcgZGF0YWZyYW1lLCBjb21iaW5lZF9wbGFudF9kZgoKYGBgCgpOb3cgdGhhdCB3ZSd2ZSBzaW11bGF0ZWQgb3VyIGRhdGEsIHdlIGNhbiBkbyBzb21lIHBsb3R0aW5nIGFuZCBzdGF0aXN0aWNhbCBhbmFseXNpcyBvbiBpdCEKYGBge3J9CiMgUGxvdCB0aGUgdGltZWNvdXJzZSBkYXRhIGZyb20gY29tYmluZWRfcGxhbnRfZGYgaW4gYSB1c2VmdWwgd2F5CiMgKGNvbG9yaW5nIGJ5IENvbmRpdGlvbiwgd2l0aCBhbiBpbmRpdmlkdWFsIGxpbmUgZm9yIGVhY2ggcGxhbnQpCgojIFRoaXMgbWF5IG5vdCBoYXZlIHdvcmtlZCBhcyB5b3UgZXhwZWN0ZWQgYmVjYXVzZSBhcyBmYXIgYXMgZ2dwbG90IGlzIGNvbmNlcm5lZCwKIyB0aGUgaW5kaXZpZHVhbHMgbGFiZWxlZCAiMSIgYXJlIGFsbCB0aGUgc2FtZSBpbmRpdmlkdWFsLCByZWdhcmRsZXNzIG9mIHdoZXRoZXIKIyB0aGV5IGNvbWUgZnJvbSB0aGUgc2FsdCBjb25kaXRpb24gb3Igbm90LiBXZSBoYXZlIHRvIHRlbGwgZ2dwbG90IHRoaXMgaXMgbm90CiMgdGhlIGNhc2UKY29tYmluZWRfcGxhbnRfZGYkcGxhbnQuaWQuZnVsbCA8LQogIHBhc3RlKGNvbWJpbmVkX3BsYW50X2RmJENvbmRpdGlvbiwgY29tYmluZWRfcGxhbnRfZGYkcGxhbnQuaWQpClZpZXcoY29tYmluZWRfcGxhbnRfZGYpCiMgVGhpcyBpcyBhICp2ZXJ5KiBjb21tb24gcHJvYmxlbSB3aXRoIGRhdGEgdGhhdCBjb21lcyBmcm9tIG11bHRpcGxlIGV4cGVyaW1lbnRzCiMgKGUuZy4gd2VsbCBsb2NhdGlvbiBpcyB1bmlxdWUgd2l0aGluIGEgOTYtd2VsbCBwbGF0ZSwgYnV0IHlvdSBydW4gaW50bwojIHByb2JsZW1zIHdoZW4gY29tYmluaW5nIGRhdGEgZnJvbSBtdWx0aXBsZSBwbGF0ZXMpCgojIHJlcGVhdCB0aGUgcGxvdCwgdXNpbmcgdGhlIHBsYW50LmlkLmZ1bGwgY29sdW1uCgpgYGAKCmBgYHtyfQojIFVzZSBsbSB0byBtb2RlbCB0aGUgZWZmZWN0IG9mIENvbmRpdGlvbiBvbiBwbGFudCBncm93dGgsIGFuZCBjaGVjayB0aGUgb3V0cHV0CiMgSXMgdGhlIG9ic2VydmVkIGNoYW5nZSBzaWduaWZpY2FudD8KCmBgYApXaGF0IHdvdWxkIHlvdSBkbyBpZiB5b3Ugd2FudGVkIHRvIHJlcGVhdCB0aGlzIHNpbXVsYXRpb24gYW5kIGNvbXBhcmlzb24gMTAwMCB0aW1lcywgc2F2aW5nIHRoZSBwLXZhbHVlIGFzc29jaWF0ZWQgd2l0aCB0aGUgZWZmZWN0IG9mIENvbmRpdGlvbiBlYWNoIHRpbWU/CgojIFVzaW5nIGNvZGluZyBpbiBzY2llbmNlCgpJIHdhbnRlZCB0byBzaGFyZSBzb21lIGZpbmFsIHJlc291cmNlcyB0aGF0IEkgdGhpbmsgYXJlIGdyZWF0IHBsYWNlcyB0byBsb29rIHRvIGtlZXAgZ29pbmcgd2l0aCBsZWFybmluZyB0byBjb2RlLgoKKiBbU29mdHdhcmVDYXJwZW50cnldKGh0dHBzOi8vc29mdHdhcmUtY2FycGVudHJ5Lm9yZy9sZXNzb25zLyk6IFNldHMgb2YgbGVzc29ucyBvbiB0aGUgY29kaW5nLCBtb3N0bHkgYXQgYW4gaW50cm9kdWN0b3J5L2Vhcmx5IGludGVybWVkaWF0ZSBsZXZlbC4gTG90cyBvZiBvdmVybGFwIHdpdGggdGhpcyBjb3Vyc2UgaW4gdGhlIFIgc2VjdGlvbnMsIGJ1dCBwcmVzZW50ZWQgaW4gYSBkaWZmZXJlbnQgd2F5OyBhbHNvIGNvdXJzZXMgaW4gUHl0aG9uLCBVbml4LCBhbmQgcmVhbGx5IHVzZWZ1bCB0b29scyBsaWtlIEdpdEh1YgoKKiBbSmVubnkgQnJ5YW4ncyBVQkMncyBTVEFUNTQ1IGNvdXJzZV0oaHR0cHM6Ly9zdGF0NTQ1LmNvbS8pOiB3YWxrcyB5b3UgdGhyb3VnaCBpbnRyby0gYW5kIGludGVybWVkaWF0ZS1sZXZlbCBSLCBleHBsYWluaW5nIG5vdCBqdXN0IHRoZSBwcm9ncmFtbWluZywgYnV0IGdvb2Qgd2F5cyB0byB0aGluayBhYm91dCBhbmFseXppbmcgZGF0YSBhbmQgY29kaW5nIHRoZSBhbmFseXNpcy4gSWYgeW91IHdhbnQgYSBtb3JlIGluLWRlcHRoIGNvdXJzZSBhYm91dCBSIGluIGdlbmVyYWwsIEkgaGlnaGx5IHJlY29tbWVuZCB0aGlzLgoKSWYgeW91J3JlIHBsYW5uaW5nIHRvIHNwZW5kIGEgbG90IG9mIHRpbWUgY29kaW5nIChpLmUuIG5vdCBqdXN0IGEgb25lLW9mZiBhbmFseXNpcywgYnV0IG1vcmUgbG9uZy10ZXJtIHByb2plY3RzKSwgdGhlIGZvbGxvd2luZyBhcnRpY2xlcyBhcmUgZ3JlYXQgc291cmNlcyBmb3IgdGhpbmtpbmcgYWJvdXQgd2hhdCB0byBhc3BpcmUgdG8uIFRoZXkgY2FuIGJlIGRhdW50aW5nIGlmIHlvdSB0cnkgdG8gYWNoaWV2ZSBhbGwgdGhlaXIgc3VnZ2VzdGlvbnMgYXQgb25jZSwgYnV0IEkgdGhpbmsgYXMgdGhleSBib3RoIHRyeSB0byBwb2ludCBvdXQsIGEgYmV0dGVyIGlkZWEgaXMgdW5kZXJzdGFuZGluZyB3aGF0IGdvb2QsIHJlcHJvZHVjaWJsZSBjb2RpbmcgbG9va3MgbGlrZSBhbmQgdGhlbiBzbG93bHkgYnVpbGRpbmcgbW9yZSBhbmQgbW9yZSBvZiB0aGVzZSBwcmFjdGljZXMgaW50byB5b3VyIHdvcmsgYXMgeW91IGdyb3cuIEZldyBjb2RpbmcgcHJvamVjdHMgaW4gYmlvbG9neSBtYW5hZ2UgdG8gYWNoaWV2ZSAqZXZlcnl0aGluZyogb3V0bGluZWQgaGVyZSwgYnV0IHRoZXNlIGFydGljbGVzIHByb3ZpZGUgYW4gZXhjZWxsZW50IGxvbmctdGVybSByb2FkbWFwIGFzIHlvdSBiZWNvbWUgYSBiZXR0ZXIgY29kZXIuCgoqIFtQYXJrZXIsIEhpbGxhcnkuICgyMDE3KSBPcGluaW9uYXRlZCBBbmFsYXlzaXMgRGV2ZWxvcG1lbnQgKlBlZXJKIFByZXByaW50cyA1OmUzMjEwdjEqXShodHRwczovL3BlZXJqLmNvbS9wcmVwcmludHMvMzIxMC8pCgoqIFtXaWxzb24sIEdyZWcgKmV0IGFsLiogKDIwMTcpIEdvb2QgZW5vdWdoIHByYWN0aWNlcyBpbiBzY2llbnRpZmljIGNvbXB1dGluZy4gKlBMT1MgQ29tcHV0YXRpb25hbCBCaW9sb2d5IDEzKDYpOiBlMTAwNTUxMCpdKGh0dHBzOi8vZG9pLm9yZy8xMC4xMzcxL2pvdXJuYWwucGNiaS4xMDA1NTEwKQoK