How to simulate a bouncing ball in R - more on simecol events and external inputs

In this post I'll clarify the incorporation of external inputs in the dynamic model specification using simecol, again reviewing the relevant parts of the simecol simulation model for the Battle of Iwo Jima that I presented in the last post. I will also give an alternate specification for this model and compare both ways of specifying external inputs in view of execution times.

For the second part of this post, I'll be using another example - a really simple one of a bouncing ball - to illustrate on how to use events in your dynamic system models using simecol.

Iwo Jima revisited - how to specify external inputs for dynamic simecol models

You may recall the last post in which we implemented a model for the battle of Iwo Jima between the American (A) and Japanese (J) armies based on the following Lanchester equation:

\[\begin{align*}\dfrac{dA}{dt} &= f(t) - j \cdot J \cdot J \\ \dfrac{dJ}{dt} &= -a \cdot A \end{align*} \]

The external inputs function, representing the number of American reinforcements and the days when they were received, was the following:

\[f(t) = \left\{ \begin{array}{cl} 54000 & 0 \leq t \lt 1 \\ 0 & 1 \leq t \lt 2 \\ 6000 & 2 \leq t \lt 3 \\ 0 & 3 \leq t \lt 5 \\ 13000 & 5 \leq t \lt 6 \\ 0 & 6 \leq t \leq 36 \end{array} \right.\]

One aspect that would often confuse my students when I gave this class is the following: given the external inputs function above and the differential equation describing the change of American troops over time, does this mean, for example, that in each time instant from \(t=0\) to \(t \lt 1\) (there could be a great many deal of those), the Americans were receiving 54000 troops? Obviously not, but that question is answered if you look at the left-hand side of differential equation describing the change in American troops over time. For example, during the first day (\(t\in[0,1)\)), the change in the number of American troops at each infinitesimally small time step would be \(dA=\left(54000 - j\cdot J\right)\cdot dt\). Notice that while the 54000 increment in troops within the parentheses in the right hand expression looks like an awful lot, it becomes a small increment when multiplied by an infinitesimally small time step \(dt\).

We specified this external inputs function in the "inputs" slot of the iwojima odeModel, which I transcribe here once more for review:

iwojima1 <- new("odeModel",
    main = function (time, init, parms, ...) {
        x <- init
        p <- parms
        f <- approxTime1(inputs, time, rule = 2)["reinforcements"]
        dame <- f - p["j"] * x["jap"]
        djap <- - p["a"] * x["ame"]
        list(c(dame, djap))},
    parms = c(j=0.0577, a=0.0106),
    times = c(from=0, to=36, by=0.01),
    init = c(ame=0, jap=21500),
    inputs = as.matrix(data.frame(
                day = c(0, 0.999, 1, 1.999, 2, 2.999, 3, 4.999, 
                        5, 5.999, 6, 36),
                reinforcements = c(54000, 54000, 0, 0, 6000, 6000, 0, 0, 
                            13000, 13000, 0, 0))),
    solver = "lsoda") 

In the code above, the input slot given in lines 12 to 16 of the code above contains a data frame with two columns: "day" and "reinforcements". We can see that this somehow maps to the reinforcements function of the iwojima model, where, for example, we had 54000 reinforcements coming in from \(t=0\) to \(t=1\), but why do we include the function value at time instants like 0.999, 1.999, 2.999, 4.999 and 5.999?

The reason for doing this is that we shall linearly interpolate this "function" (which is not a function, but a data frame really). So anywhere from \(t=0\) to \(t=0.999\), the function's value will be \(f(t)=54000\), but at \(t=0.9995\), for example, the value would be halfway between 54000 and 0, that is, \(f(0.9995)=27000\). Thereafter, it quickly drops to 0, since there were no reinforcements for the next day.

The data frame is linearly interpolated by the approxTime1 function in line 5 of the code above. This simecol function takes as arguments the time (the time instant as per the simulation clock) and inputs data frame given in lines 12 to 16. The rule argument is 2 because if time is anywhere outside of the two extreme time instants being considered, we want it to be set to the value at the closest time extreme (see the help file for approxTime1), although this is not necessary since we won't be running the simulation past day 36. By the way, if we look at the help file for approxTime1, we read that although this function is less flexible than approxTime as it takes only a single value and not a vector for the xout argument, it is more efficient in terms of execution time if the x argument is a data frame.

We could alternatively specify the iwojima model as follows:

iwojima2 <- new("odeModel",
    main = function (time, init, parms, ...) {
        x <- init
        p <- parms
        dame <- f(time) - p["j"] * x["jap"]
        djap <- - p["a"] * x["ame"]
        list(c(dame, djap)) },
    equations = list(
        f = function(t) {
            if (t<1) return(54000)
            else
              if (t<2) return(0)
              else
                if (t<3) return(6000)
                else
                  if (t<5) return(0)
                  else
                    if (t<6) return(13000)
                    else return(0)}), 
    parms = c(j=0.0577, a=0.0106),
    times = c(from=0, to=36, by=0.01),
    init = c(ame=0, jap=21500),
    solver="lsoda")

In this second version of the iwojima odeModel object, we include the external inputs function in the equations slot and not in the inputs slot as we had previously done. This second version, although it has more lines of code, is more tractable to our intuitive grasp, since the reinforcement function is defined as we would normally define a step-wise function in R and not by some roundabout process such as defining a data frame with weird time-instant specifications that we later interpolate linearly. However, the crucial point is not whether the code looks prettier or not, but which model specification is more time-efficient in terms of execution. The reason we are concerned about this is that since we are to fit the model's parameters, this will involve repeated simulations of our model done by the simecol fitOdemodel function. Using the following code we clock both fittings:

#we read the historical data...
obs <- read.csv2("iwo_jima.csv")
obsv <- which(!(is.na(obs$jap)))
#the weightdf dataframe assigns equal weight to all observations,
#except for the NA values in the japanese column (those get 0 weight)
weightdf <- data.frame(ame=rep(1, nrow(obs)), jap = rep(0, nrow=(obs)))
weightdf[1,"jap"] <- 1
weightdf[nrow(obs),"jap"] <- 1
#and now we fit the model parameters with simecol's fitOdeModel
#including our weightdf as weights makes the PORT routine faster
system.time(
  result_fit1 <- fitOdeModel(iwojima1,
    obstime=obs$time,
    yobs=obs[2:3],
    fn=ssqOdeModel,
    weights=weightdf,
    method="PORT",
    scale=c(1/0.1, 1/0.01),
    lower=c(j=0,a=0))
)
system.time(
  result_fit2 <- fitOdeModel(iwojima2,
    obstime=obs$time,
    yobs=obs[2:3],
    fn=ssqOdeModel,
    weights=weightdf,
    method="PORT",
    scale=c(1/0.1, 1/0.01),
    lower=c(j=0,a=0))
)

The execution time for fitting the iwojima1 model (using the interpolated inputs data frame) is 173.328 seconds on my CPU, while that of fitting the iwojima2 model (using a stepwise function definition in the equations slot) is ... 24.312 seconds! So in this particular case, pretty and efficient coincide1. Another surprising result is that the fits were not quite the same. For the iwojima1 model, the sum of squares is 0.05121009, while for the iwojima2 model, it was slightly higher at 0.05363194. I don't know if this is always the case (using the interpolated inputs data frame results in a more accurate fit to the data). More testing would have to be done to see if the gain in goodness of fit is significant and if it justifies the much slower execution time. But then again, we have not taken into consideration the problem of overfitting to the observed data2, which would further complicate this discussion.

Using simecol to simulate a bouncing ball

The reason we're using a simecol simulation of a bouncing ball as an example is to illustrate another characteristic of odeModel objects in simecol - simulation events, which occur whenever the value of a state variable changes abruptly in a manner not stipulated by the differential equations dictating the internal dynamics of the system. In a certain sense, the external inputs function considered previously can be thought of as an event. However, in the case of the bouncing ball, we have another type of event that occurs when the ball hits the ground and bounces back up, thereby changing the sign of its velocity. Of course, for added realism, we not only change the sign of the velocity, but at each bounce, we multiply the velocity by 0.9 to account for the loss of kinetic energy due to friction3, so that eventually the ball will settle on the ground. We illustrate how this is done in the following code:

#------------------------------------------------------------------
# Bouncing ball simulation using simecol
# Thomas Petzoldt - 04/09/2010
#------------------------------------------------------------------

library(simecol)
ballode <- new("odeModel",
    main = function (t, y, parms) {
        dy1 <- y[2]
        dy2 <- -9.8
        list(c(dy1, dy2))   },
    parms = 0,
    times = seq(0, 37.5, 0.25),
    init = c(height = 20, v = 0),
    solver = function(t, y, func, parms) {
        root <- function(t, y, parms) y[1]
        event <- function(t, y, parms) {
            y[1] <- 0
            y[2] <- -0.9 * y[2]
            return(y)   }
      lsodar(t, y, func, parms = NULL,events = list(func = event, 
        root = TRUE), rootfun = root)}
)

ballode <- sim(ballode)

# Animation
o <- out(ballode)
png("ball%04d.png", width=300, height=300)
for (i in 1:length(times(ballode)))
    plot(1, o[i, 2], main="", ylab="", xlab="", xaxt="n",
       col="red", cex=2, pch=16, ylim=c(0,20))
graphics.off()
#Convert the *.png image sequence into an animate .gif file
#(Note: requires the ImageMagick command line programs)
system("convert -delay 5 *.png bouncing_ball.gif")

I would first like to comment that while we have only one apparent state variable (the height) and the rest would just be the first and second derivatives of that variable (accounting for the velocity and acceleration of the ball, respectively), in the odeModel specification we work with two state variables: the height and the vertical velocity. You can see that in line 9 of the code, the height increment is the velocity, while in line 10, the velocity increment is the acceleration, which is a constant equal to \(9.8\,\tfrac{m}{s^2}\).

The new action occurs within the solver slot specification of the odeModel. Instead of using rk4, lsoda or some such predefined integrator function in that slot, we define our own function to include a root function that is triggered when an event occurs. You see, the event "ball hits the ground" can be characterised mathematically as the point in which the height state variable equals zero4, so the root function would be the height state variable itself (in line 16 of the previous code). In the event function (lines 17 to 20), we simply set the height to zero and change the direction of the bouncing ball by multiplying the velocity by a negative number (again, differing discussion of the 0.9 factor for later in this post). Finally, we use lsodar as our solver, which is a essentially the same lsoda integration routine with added capabilities for dealing with events given by root functions. In the lsodar function call, we pass our root function and event functions as a list. We could add more events (for example, if we had a ceiling and had to deal with the ball bouncing off the ceiling) by simply adding more root/event function pairs in that list.

And now, Let's Get Physical5

This bouncing ball example is a perfect pretext to use R/simecol simulations in the physics classroom. Imagine we had no knowledge of Newtonian physics but we had a fairly good grasp of the concepts of velocity and positions as variables in a physical model. Instead of presenting the simulation model directly, we could first raise a discussion about the different state variables and the differential equations describing their behavior, after which we would arive at something like the speed/aceleration model above. Instead of using the predetermined 9.8 constant as aceleration, we could infer this ourselves from observational data by using the fitOdeModel simecol function, but first, we'd have to get some data...

At this point, I'd invite a student who's good at video editing to make a movie of a bouncing ball side-by-side with a ruler to measure its height. I would ask that student to obtain a sample of png frames from that video - you don't need all the frames, just some. If filenames of those png frames have numbers identifying the frame number in the movie, then obtaining the time instants is as easy as dividing the frame number by 30 (if the movie is 30 frames per second, example). To obtain the height measurements, I would ask another student to visually check the height of the ball against the ruler in each frame. Tabulating these values in a column of time (in seconds) and a column for height (converting the measurements into meters to keep consistent with the MKS measurement system) would give us our data frame of observations against which we would fit the model.

We could then adjust the gravitational aceleration constant and verify if it's really close to 9.8. While we're at it, we could also include a parameter for the 0.9 value corresponding to the bounce-back kinetic energy loss and set it initially to 1, to verify what it's real value is. In physics, this constant would correspond to the elasticity of the ball and the amount of ball surface-area that touches the ground when the ball is deformed while bouncing. Most kinetic-energy loss, I would observe, is due to friction and in friction energy is lost as heat, but we could have some discussion in class about that.

If we wanted to get more sophisticated in our modeling, we could also include a component to model the air-drag of the ball in its trajectory up and down. This would be a realistic addition to the model, as our bouncing ball was most likely not filmed in a vaccum. All of these additions to the models would entail re-fitting the model to the data obtained from the film. At each new fit, we could observe the goodness of fit sum of square values to see what accuracy we gained in our model.

If I was a physics teacher (which I'm not), this is the sort of thing I'd like to do in class. If system dynamic modeling principles were to be taught across the curriculum, students could just apply this to the study of physics, biology, social studies and other topics to deduce and re-invent by themselves all these laws and principles which they would otherwise have to memorize. This, no doubt, would make the learning more significant and meaningful. The great Isaac Newton, whom we could probably consider the great grand-father of System Dynamics, is quoted as saying that if he could see farther than others, it was because he stood on the shoulders of giants. What fun Newton would have had if he had a computer with R and simecol to play with! Then again, maybe its better that way because he wouldn't have invented Differential Calculus6.

Notes

As a general rule, more elegant and nicer looking code is more efficient in terms of execution time.
Overfitting is an important topic in the machine learning context. When we fit a model to observed data, we want the in-sample error to be as small as possible. One way to achieve this is by having more complex models incorporating a greater number of parameters. However, there's a penalty to pay for this, because as we incorporate more parameters into our model, we will almost surely fit the errors that are present in our sample of observations accounting for the peculiarities of that particular sample. In other words, overfitting occurs when we fit the noise in our data sample due to having more parameters in our model and this, while it guarantees a lower in-sample error, will almost surely result in a larger out-of-sample error, which is the error we're really interested in reducing. This is known as the principle of Occam's Razor.

In our case, we don't have a different number of model parameters in either model and the observed differences in goodness of fit may not even constitute a case of overfitting at all. But if we think about it, the linear interpolation of the data frame mechanism would be more true to life to what occurred in the battle field. In the stepwise function implementation (using the equations slot), you suddenly have the stream of incoming reinforcements cut short to 0 as the simulation transitions from \(t\lt1\) to \(t\geq 1\), whereas in the linear interpolation of the data frame implementation, you have a more gradual reduction in the reinforcement stream. The former, being more true to real-life conditions, is probably why we got a better goodness of fit. I hope you followed me...
We will later in this post go into details as to the reasons for modeling the loss of kinetic energy by multiplying the velocity by a factor of 0.9
... Or is almost equal to zero, depending on the error tolerance you define (or predefine in the lsodar function call).
Sorry about the cheesy reference to that Olivia Newton-John 1980's song. Anyhow, when looking up the Wikipedia article on her while writing this note, I discovered two interesting facts about her ancestry. On the maternal side, she is descended from Max Born, the famous Nobel Prize atomic physicist. Olivia Newton-John's father was involved in the Enigma project at Bletchley Park, in which the allied mathematicians cracked the Nazi ciphers in World War II. Both facts are tangent on the sort of "physical" that I want to talk about in this section, so let's get physical...
The quote that Newton (not to be confused with Newton-John of "Physical"-fame) phrased in English - nanos gigantum humeris insidentes - is actually ascribed to Bernard de Chartres in the 12th Century. As to the invention of Differential Calculus, there's also a view according to which Leibniz invented it independently, but this is too great of a dispute to go into in this footnote.

Bibliography

Olivia Newton-John. (2016, August 9). In Wikipedia, The Free Encyclopedia. Retrieved August 14, 2016, from https://en.wikipedia.org/wiki/Olivia_Newton-John.
PETZOLDT, T. and RINKE , K. (2007). "simecol: An Object-Oriented Framework for Ecological Modeling in R". Journal of Statistical Software, 22(9), pp. 1–31. ISSN 1548-7660. http://www.jstatsoft.org/v22/i09.
R DEVELOPMENT CORE TEAM (2009). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0. http://www.R-project.org.
Standing on the shoulders of giants. (2016, August 7). In Wikipedia, The Free Encyclopedia. Retrieved August 14, 2016, from https://en.wikipedia.org/wiki/Standing_on_the_shoulders_of_giants.