Tag Archives: NASA

Are There PID Controllers in Data Science?

A while back I went to a data science meetup, and the presentation was on how an analyst solved his problem with a PID controller. As an engineer I’m well familiar with PID controllers. I spent a lot of time studying them in school. The problem in this scenario was optimizing the amount and cost of ad impressions in a given day. My first thought when I saw it was that this isn’t data science. You’d think that with the amount of data available that a regression or decision tree might work better. I’ve thought about this quite a bit since then, and my view has changed. In fact, I now his approach is brilliant in its simplicity.

PID Controller

By TravTigerEE of Wikepedia

For those not familiar, a Proportional Integral Derivative controller is a control-feedback loop that automates the control of systems. That’s a fancy way of saying that they control an input and monitor the output. If there’s a difference between the output and the desired output, the input is adjusted to account for this error. The first designs were for steering a ship, automating the task of maintaining a heading when there are wind or currents to account for. These aren’t new, the first designs were completed in the 1920’s, and today they’re commonplace. In fact, you’ve used them.

Your home’s thermostat is a PID controller. The input is whether or not to turn on your air conditioner or heater, the desired output is the temperature you want, and it measures the inside temperature. Most cars these days have cruise control, that’s another example of a PID controller that we use on a daily basis. An aircraft autopilot is a more complex example, as it maintains heading, altitude and airspeed. Your computer’s hard drive uses a PID controller to position its head on the platter. And they’re used extensively in manufacturing and industrial applications.

Some of the more cool or exotic uses of PID controllers are in a car’s suspension system, to make the ride ultra smooth. NASA uses them on their launch platforms to aid in keeping rockets balanced upright. Recently SpaceX and Blue Origin have autonomously landed rockets through the use of advanced control systems. And Boston Dynamics has created a number of awesome walking robots that maintain balance across slippery or rugged terrain. Some of these applications are quite a bit more advanced than the simple PID controller, but they definitely contain PID controllers.

By now hopefully you can see why I’d question whether PID controllers would be considered a data science tool. Certainly they’re useful, and if you find an application where they’ll work then its a great choice because they’re well documented, relatively easy to implement, and once they’re tuned they require little maintenance. These are all great attributes, and yet I wouldn’t expect them to be covered in a statistics course. Or a computer science course. The math behind PID controllers is calculus.┬áIn my own education, I studied a lot of calculus and these were only covered in my engineering coursework.

In fact, if we look at common data science tools, PID controllers are absent. One of the great reasons for choosing R over another language is that there are packages for everything. A Johns Hopkins professor even created a popular video proclaiming There’s an R Package for That, playing on the popular cell phone map ads from a few years ago. And yet, if you do a search there is no PID controller package for R. There is a blog post showing how to create one, but no package. And what about Python’s data science library scikit-learn? Nope. Data science upstart Julia? No. Not even Apache’s Hadoop or Spark frameworks have pre-built algorithms for a PID controller.

These data science packages and libraries are all designed to make sense of thousands (or more) rows of hundreds of variables. A PID controller reads one thing and controls another. PID controllers don’t work with big data. And yet, they’re incredibly useful in applications where we work with data.

So now lets pay closer attention to the data science side. A popular post over the past few years has been Drew Conway’s data science Venn diagram. One of the 3 primary components is hacking skills. Conway partly explains this as discovery and building knowledge via hypotheses and experimentation. That means figuring out how to get things done. You aren’t limited to what libraries have to offer. If you have domain experience and can simplify your problem to the point that a PID controller accomplishes what you need, that’s valuable no matter what you call it. I’d say that fits within data science.

It has also occurred to me that there are a few other data science applications not covered universally by data science tools. Collaborative filtering and A/B testing are both definitely in the realm of data science, and yet if you choose one of these methods for your project you’ll likely be coding up your own solution. Natural language processing is another example of a field within data science that calls on skills outside of a statistical model. There are definitely packages and libraries for NLP in the major languages, but this field has its own methods outside of categorization, classification or regression. And maybe that’s my point. This is data science. You do what you have to do.

Notice that I’m careful to call this data science. Another closely related moniker is machine learning. A PID controller is definitely not machine learning, nor knowledge discovery nor operations research nor data mining. This is not a method for classification or clustering. There is no model being created. This isn’t Frequentist or Bayesian. PID controllers can definitely be useful for solving data science problems, but they cannot predict anything.

What are some potential use cases where a data scientist might choose a PID controller? As a generic answer I’d say anywhere that you believe there’s a direct relationship between a variable that you control and an output that you monitor. The data stream should be continuous, or at least assumed continuous. The example that lead to this thought is perfect, controlling the number of ad impressions by adjusting the bid price in an auction. In this example, your ads are up and running 24 hours a day, 7 days a week, and you know almost instantly how many impressions you earned.

I actually think PID controllers could work well for a lot of problems with the online realm. Buying online ads, budget pacing and load balancing come to mind. There are probably areas within SEO where it would help.

This would not work as well for optimizing football game attendance by adjusting ticket prices. In this scenario the data is not continuous, there are days or months between events. There are also too many other variables at play. If temperatures are below freezing then ticket price probably isn’t a factor in game attendance, people won’t be as willing to sit in the cold. If the game is post-season then temperature probably isn’t a factor, people will attend regardless of temperature because they’re excited to see their team in the playoffs. These are guesses, but hopefully you see my point.

My own conclusion in this is that, yes, PID controllers are fair game within the realm of data science. The realm of controllers is stable, mature, and is still an area of active research. If you find a scenario where a PID controller gives you the results you’re looking for, then its absolutely a good choice.

Gene Kranz

Recent photo of NASA's very own Gene Kranz.
Image via Wikipedia

Tonight Gene Kranz, the famed NASA mission controller played by Ed Harris in Apollo 13, came to speak at UT and I was able to make it. I went with low expectations because I didn’t know much about him other than what I had seen in the movie, and just wow. This man is awesome. The speech was really inspiring and gave me renewed pride in being an engineer and being an American. It was that kind of speech.

Before Kranz came out, one of the professors involved in bringing him to speak at UT introduced some of the students and people present in the audience, and he did one of the coolest things that I’ve seen at any speech. He had the veterans stand to be recognized, and then he asked the military members awaiting a commission stand and be recognized. I’m proud of America, and I’m proud of my service, and I think its awesome that he gave credit to the military’s future leaders.

Mr. Kranz spoke for over an hour, most of it spent walking us through his thoughts and decision process during the Apollo 13 mission. Based on what he said I believe that the movie is pretty accurate (I’ve seen it dozens of times). What really came through in his speech, that I hadn’t noticed in the movie, is the amount of trust and cooperation between the team members. He pointed out a couple of times when he made a decision in the interest of moving forward, that some of the other engineers disagreed with, but once Kranz announced the decision the whole team got on board and worked together for the benefit of the mission. The whole story is inspiring, but hearing it from the perspective of this man in particular is something that I won’t soon forget.

When he was done he got a standing ovation, which he was apparently uncomfortable with because he quickly had us sit back down for a question and answer session. This was my favorite part of the event, the questions were great and he had some very good answers. I don’t remember all of the questions, but here are the ones that struck me.

He was asked if he was pleased with his portrayal in movies. He said that he thought Ed Harris did a great job, and that the story line of Apollo 13 was pretty accurate, with one notable exception. In the scene just after they realize the extent of the problem a few of the engineers state that such and such can’t be done. In the movie Harris (playing Kranz) blows up and says something along the lines of “failure is not an option”. Tonight Kranz said that he would never blow up at anyone who was within his authority, he wouldn’t allow himself the luxury of emotion or losing his cool when leading his team. I thought that was particularly insightful.

Someone asked him what he thought of NASA’s decommissioning of the space shuttle and planned absence from spaceflight. He said he thinks this is a travesty because America needs awesome challenges to inspire and bring out the best in its people. Leaving a gap in the flight plan also means forgetting things that we’ve fought hard to learn.

When asked what he thought of privatized spaceflight, he said “its the greatest thing since canned beer.” There is nothing like advancing technology to stimulate the economy and capture the attention of the world.

My favorite response came when he was asked how he found or acquired the leadership skills to bring out the absolute best in his team. He said that he grew up in a time when his father’s generation fought in WWII. He and his peers were too young to go, but looked at his parent’s generation as heroes which challenged him to achieve something great. Then in the early days of supersonic flight men like John Stapp, Joe Kittinger and Chuck Yeager risked their lives for advances in aviation technology. During Project Mercury John Glenn climbed into an Atlas rocket, and at that time three of the first five of them had exploded on launch. Everyone at NASA had a clear understanding of the risks involved and chose to participate anyway because we saw the greatness within the goals. This brings a very clear focus that is difficult to achieve in any environment. Kranz said that he’s spent a lot of time ruminating over his career and isn’t able to point to any one thing that makes him a leader, but that he does recognize that he wanted to live up to the greatness that he saw in his father’s generation and believed that he could.

I just can’t express how awesome it was to hear Kranz speak. His involvement in Apollo 13 and NASA makes him an American icon, and his story is definitely inspiring. I was tempted to skip because I had homework to get done last night. I’m really glad that I didn’t.

Burt Rutan Speech

In the process of doing some research for a project on campus, I came across this speech that Burt Rutan made back in Feb of 2006. Rutan has been a hero of mine for a long time. This speech is pretty encouraging, and exciting.

I don’t agree with every single point that he makes, but its still pretty cool to me that he’s trying to ignite people’s passion about advancing aviation technology and space exploration. And, I do think that the private sector will do a better job with safety than NASA has done.

Our kids will have an opportunity to go on a sub-orbital flight within their lifetime, just for fun. That is incredible. I want to be part of that.

Dr. Griffin

This afternoon Dr. Michael Griffin, NASA‘s administrator, came to speak at UT. I was fortunate enough to be able to hear him, and it was fantastic.

He spent 30 minutes laying out his vision for the next 30 years of space exploration. I won’t go into the details, but his plan includes a large amount of lunar exploration, during which time we’ll be honing our skills and technologies, and then use that knowledge to pursue Mars. I have to say I think his plan is wise.

His vision includes opportunities for entrepreneurs to support NASA’s projects in a variety of roles. This means that Blue Origin, SpaceX and others will have a part to play in NASA’s plan. This is the same agency which effectively squashed Beal Aerospace a few years ago, so this is fantastic news!

I thoroughly enjoyed his presentation. He said a lot of things that I found exciting, but there was one salient quote. A professor asked a question that prompted Griffin to talk about education.

NASA’s role in education is to pursue technologies that are exciting enough to motivate kids to stick with the hard subjects and pursue technical degrees so that they can come participate in these thrilling projects. We have not done a very good job of that for the past 35 years.

I wanted to stand up and start singing! President Bush did a great thing when he appointed an engineer (Griffin holds no less than 5 advanced degrees in engineering) to this position. He’s the right man for the job. I just can’t say enough good things about Dr. Griffin.

I’ve been saying for a while that I don’t want to go to work for NASA. Dr. Griffin changed my perspective. If he’s still the one in charge when I graduate, I’ll submit my resume.

NASA’s Mistake

Um, I can’t let this go without comment. Yesterday NASA’s administrator, Michael Griffin, was interviewed by USAToday. In that interview, he admits that the Space Shuttle and the International Space Station are mistakes. I completely agree.

This is the best move NASA has made in 30 years. The Space Shuttle program is a failure. The program has launched just over 100 times, and they’ve lost two craft. Would you get in a car that blew up once every fifty times you drove it? By contrast, Russia has continued to use the same technology since their space program’s inception over 40 years ago, and they’ve never lost a single ship!

Last weekend I was visiting with some folks and someone quipped in a disapproving voice that we’re going back to the moon using 40 year old technology. I kept my mouth shut, but it was tough for me. I wanted to yell, “have you even paid attention over the past 20 years!?!?”

Merging the best of shuttle and Apollo technology is absolutely the right direction for NASA. I’ll say more about going back to the moon in another post, but I couldn’t let this go.

See, while NASA has been putting time, money & energy into the shuttle and space station programs, other countries have started their own space agencies. There’s more than 30 of them around the globe. Oh, sure, you’ve heard of the Chinese and Japanese space agencies, but how about Austria, Belgium, Denmark, Italy, Spain, Norway, Sweden. Even Finland has one! Finland!

Now, there’s a lot of people at NASA that would like for you to believe that we’re working cooperatively with all of these agencies, and that’s true, partially. We’ve been working together with them on the space station. But what about planetary exploration? Um, no. We’re competing with them.

See, someone is eventually going to fully explore the Moon. And Mars. And beyond. And the first one that does gets to claim the resources that they find for their very own. Remember the exploration of the Americas? Like that, only planetary.

The good news is that NASA has experience, and a lot of the other players are still learning. This is why it makes such great sense for us to apply what we’ve learned through the shuttle to the Apollo technology from years ago.

Its all very exciting, really. If you, like me, want to keep up with what’s going on, NASA Watch is a great website which tracks what they’re up to.

Return To Space

Lots of buzz in the space realm lately. NASA announced last week that they plan to resume shuttle opps in May with a Discovery launch, and that’ll be followed by an Atlantis launch in July. This is good news for America.

NASA has taken a backseat to everyone over the past couple of years. Since the Columbia tragedy other countries have taken the lion’s share of supporting the space station. And SpaceShipOne has stolen the spotlight for the past year by winning the X-Prize competition for privatized spaceflight. And since then several companies have begun working towards taking passengers into space.

I think everyone has heard of Richard Branson’s Virgin Galactic by now. Just yesterday I learned of Rocket Plane, Inc, an Oklahoma company that plans to do the same thing. This industry has been gaining momentum for the past couple of years, so it seems likely to me that other companies will join in soon. NASA has been the world leader in space exploration for decades, it’ll be really interesting to see how they respond to this.

I’m really glad to see the shuttles flying again. It might be worth a road trip to go watch the launch in Florida. And please let me take this opportunity to remind you that I am really freakin excited about being an aerospace engineer.