Category Archives: Computing

Think Different - Upgrading a "Sad" iMac

phil-schiller-625x350

During the spring announcements, Apple executive Phil Schiller said that about 600 million people are using computers over five years old and said "This is really sad." The comment made a lot of folks angry, especially those in despair over the ever rising cost of Apple hardware with seemingly less functionality. Along with several of my co-workers (all Apple fans, including myself) there has been chatter of what the company's future holds. Today I'd like to tell the brief story of my Apple computing adventures and recent hardware upgrades I made to a 2007 iMac that have it running like a champ.

I was formerly a PC person, but during my undergraduate I was converted to Mac by the ease of developing code and doing scientific research in a *nix based environment. Macs were less prone to viruses, superior hardware that was designed to be compatible with superior software. I had no problem paying good money for this hardware, it lasted and lasted. I dove in with a white MacBook and 27" iMac in 2007. The MacBook had some cracking issues with the plastic that Apple was swift to repair for free. The iMac was flawless. I could upgrade the RAM easily. The hard drive wasn't so easy, but that's for later.

In 2011 I decided to buy a MacBook Pro laptop as my primary machine along with the beautiful 27" thunderbolt display. This was a very expensive trip to the mall! I still use the monitor daily and love it. The MacBook Pro did the majority of my computing, until late 2013 - wait what? Yes, due to a design flaw, the excessive heat from intense computation had cracked solder joints on the logic board. Others I know were plagued by this as well. It was an ~$800 repair to get back to working, but it could be expected to fail again. I set aside a several thousand dollar machine after not quite 3 years of service. I had just put a hybrid SSD in and more RAM to add insult to injury. Apple did nothing.

By this point in time I had a nice 2012 iMac at work that all of my "serious" computation happened on, so I bought a MacBook Air for my laptop/home machine. It's light and the battery life is unbeatable. I really like it, but in just a few years it has aged poorly. I'd like to get more RAM inside and a larger SSD. That's not possible since in an effort to make it thin, these components have been soldered to the logic board. There are no upgrades possible. Again... what? I can't take advantage of the rapid reduction in storage prices? Ugh.

Now to the point of the story. My wife has been using my 2007 iMac, the first one. It was getting slow and unfriendly to use, so I decided to max out the RAM (4Gb) and put in an SSD. For a couple hundred dollars in parts and an evening's worth of work, the computer runs like a champ. It's certainly not like a new machine, but it will last for several more years and after that would be a great machine for my 3D printer to use full time. It still runs Adobe Illustrator and other graphics programs. It's a perfectly usable machine. That's all because I can upgrade it. Sure, instilling the new hard drive was a PAIN. I had to pull the monitor out and ended up doing it twice because of a contact issue. But it worked and we got more life out of it!

2016-03-12 18.41.37

While in there, I noticed that one of the power supply filter capacitors was bulging and likely bad. Not a problem! This power supply and system was over-engineered. Sure the computer was a bit thicker than my new iMac, but hey - I didn't have to replace the power supply! Weight and size really count when sending things to space, but I'm pretty sure most Apple hardware stays firmly on the ground.

2016-03-12 18.41.32

What's the point of me telling you about my history of Apple purchases? Am I not biased since I am a MacBook Air, iMac, iPhone, iPad, Apple Watch owner? It's to say that I'm worried. I've always treasured Apple products because they just worked, they let me do my job with the least amount of friction. Recently I've spend more time fighting limitations of machines that I can't upgrade, battling bugs in software released to the public that Steve Jobs would have called "beta" at best. Am I ready to jump ship? No. Am I looking around? Yes.

As many of you know, I'm involved in a lot of engineering projects as well as pure geophysical research. A lot of engineering software like SolidWorks is designed to run on Windows. Some programming and hardware CAD/CAM tools I use are Windows only as well. I've generally run a virtual machine on my computers with Parallels. (Which is now a subscription model.) It is quickly becoming the case that I'll spend time in both operating systems everyday. I even bought a Microsoft Surface because the note-taking with a pen experience is so much better than Apple's offerings.

Many CAD tools are also beginning to offer Ubuntu Linux versions as well. I've even found these to be better maintained that the Mac native versions of such applications.

I used to say that I was going to build a desktop with amazing specs and run Ubuntu on it. I'd sure miss some of my Mac Apps though, things like TextExpander, OmniFocus, and OmniOutliner to name a few. I do have them on iOS as well. I'm very curious to see what happens in the next five years as things become increasing platform independent and cloud based. Will the switch between devices become seamless? Microsoft's surprise announcement about the integration of the bash shell into Windows 10 yesterday certainly showed that they are willing to cater to developers that Apple is beginning to limit. The multi-platform OneNote product is also exceeding anything Apple Notes can do.

Right now, I can't say where we'll all be in a few years. The fact that Apple used their most recent event to announce another size of screen and watch bands instead of new Mac hardware has me worried. Power users seem to be a non-priority for Apple since the over-priced and non-upgradable Mac Pro (trashcan version) was released in 2012. I need reliable, strong computers and I'm getting tired of carrying 2 laptops, a tablet, a phone, an iPad, etc. I'm hoping the situation becomes more clear in a few years when it will be time to retire a lot of my current hardware.

A final sidenote - I recent received an 8Tb hard drive for review. In the days of the tower Mac Pro I could have popped it into one of the many equipment bays, maybe added a new video card while I was there. Now I need to buy an external dock and takeup a USB-C port. Maybe I should look into NAS systems... Maybe I should build a tower at my workbench. How have you been thinking about your computing environment recently? I'm wondering if it's time to think different.

Debugging - Book Review

51babHMIFWL

I end up doing a lot of debugging, in fact every single day I'm debugging something. Some days it is software and scripts that I'm using for my PhD research, some days it is failed laboratory equipment, and some days it's working the problems out of a new instrument design. Growing up working on mechanical things really helped me develop a knack for isolating problems, but this is not knowledge that everyone has the occasion to develop. I'm always looking for ways to help people learn good debugging techniques. There's nothing like discovering, tracking down, and fixing a bug in something. Also, the more good debuggers there are in the world, the fewer hours are waisted fruitlessly guessing at problems.

I'd heard about a debugging book that was supposed to be good for any level of debugger, from engineer to manager to homeowner. I was a little suspicious since the is such a wide audience, but I found that my library had the book and checked it out; it is "Debugging: The 9 Indispensable Rules for Finding Even the Most Elusive Software and Hardware Problems" by David Agans. The book sat on my shelf for a few months while I was "too busy" to read it. Finally, I decided to tackle a chapter a day. The chapters are short and I can handle two weeks of following the book. Each morning when I got to work, I read the chapter for that day. One weekend I read an extra chapter or two because they were enjoyable.

I'm not going to ruin the book, but I am going to tell you the 9 rules (it's okay, they are also available in poster form on the book website).

  1. Understand the System
  2. Make It Fail
  3. Quit Thinking and Look
  4. Divide and Conquer
  5. Change One Thing at a Time
  6. Keep an Audit Trail
  7. Check the Plug
  8. Get a Fresh View
  9. If You Didn't Fix It, It Ain't Fixed

They seem simple, but think of the times you've tried to fix something and skipped around because you thought you knew better to have it come back to sting you. If you've done a lot of debugging, you can already see the value of this book.

The book contains a lot of "war stories" that tell tales of when a rule or several rules were the key in a debugging problem. My personal favorite was the story about a video conferencing system that seemed to randomly crash. Turns out the compression of the video had problems with certain patterns and when the author wore a plaid shirt to work and would test the system, it failed. He ended up sending photocopies of his shirt to the manufacturer of the chip. Fun stories like that made the book fun to read and show how you have to pay attention to everything when debugging.

The book has a slight hardware leaning, but has examples of software, hardware, and home appliances. I think that all experimentalists or engineers should read this early on in their education. It'll save hours of time and make you seem like a bug whisperer. Managers can learn from this too and see the need to provide proper time, tools, and support to engineering.

If you like the blog, you'll probably like this book or know someone that needs it for Christmas. I am not being paid to write this, I don't know the author or publisher, but wanted to share this find with the blog audience. Enjoy and leave any comments about resources or your own debugging issues!

Using Visual Mics in Geoscience

Image: TED Talk

Image: TED Talk

Last time I wrote up the basics of a tip sent in by Evan over at Agile Geoscience. This technology is very neat, if you haven't read that post first, please do and watch the TED talk. This post is going to be about how we could apply this to problems in geoscience. Some of these ideas are "low hanging fruit" that could be relatively easy to accomplish, others are in need of a few more PhD students to flesh them out. I'd love to work on it myself, but I keep hearing about this thing called graduation and think it sounds like a grand time. Maybe after graduation I can play with some of these in detail, maybe before I can just experiment around a bit.

In his email to me, Evan pointed out that this visual microphone work IS seismology of sorts. In seismology we look at the motion of the Earth with seismometers or geophones. If we have a lot of them and can look at the motion of the Earth in a lot of places over time, we can learn a lot about what it's like inside the Earth. This type of survey has been used to understand problems as big as the structure of the Earth and as small as finding artifacts or oil in shallow deposits. In (very) general terms we look at very low frequency waves for Earth structure problems with periods of a second to a few hundred seconds. For more near surface problems we may look at signals up to a few hundred cycles per second (Hz). Remember in the last post I said that we collect audio data at around 44,200 Hz? That's because as humans we are able to hear up to around 20,000 Hz. All of this is a lot higher frequency than we ever use in geoscience... I'm thinking that makes this technique somewhat easier to apply and maybe even able to use poor quality images.

So what could it be used for? Below are a few bullet points  of ideas. Please add to them in the comments or tear them apart. I agree with Evan that there is some great potential here.

  • Find/visualize/simulate stress and strain concentration in heterogeneous materials.
  • Extract modulus of rock from video of compression tests. Could be as simple as stepping on the rock.
  • Extend the model to add predicted failure and show expected strain right before failure.
  • Look at a sample from multiple camera views and combine for the full anisotropic properties. This smells of some modification of structure from motion techniques.
  • Characterize complicated machines stiffness/strain to correct for it when reducing experimental data without complex models for the machine.
  • Try prediction of building response during shaking.
  • What about perturbing bodies of water and modeling the wave-field?

With everything in science, engineering, and life, there are tradeoffs. What are the catches here? Well, the resolution is pretty good, but may not be good enough for the small differences in properties we sometimes deal with. In translating this over to work on seismic data I think a lot of algorithm changes would have to happen that may end up making it about the same utility as our first-principles approaches. A big limitation for earthquake science is what happens at large strains. The model looks at small strains/vibrations to model linear elastic behavior. That's like stretching a spring and letting it spring back (remember Hooke's Law?). Things get interesting in the non-linear part of deformation when we permanently deform things. Imagine that you stretch the spring above much further than it was designed to be. The nice linear-elastic behavior would go away and plastic deformation would start. You'd deform the spring and it wouldn't ever spring back the same way it was again. Eventually, as you keep stretching, the spring would break. The non-linear parts of deformation are really important to us in earthquake science for obvious reasons. For active seismic survey people, the small strain approximation isn't bad though.

Another issue I can imagine is combining video from different orientations to recover the full behavior of the material. I don't know all of the details of Abe's algorithm, but I think it would have problems with anisotropic materials. Those are materials that behave differently in different directions. Imagine a cube that can be easily squeezed on two opposing faces, but not easily squeezed on the others. Some rocks behave in such a way (layered rocks in particular). That's really important since they are also common rocks for hydrocarbon operations to target! Surrounding the sample area with different views (video or seismic) and using all of that information should do the job, but it's bound to be pretty tricky.

The last thing that strikes me is processing time. I don't think I've seen any quotes of how long the processing of the video clips took to recover the audio. While I don't think it's ludicrous, I think the short clips could conceivably take a few hours per every 10 seconds (this is a guess). For large or long duration geo experiments that could become an issue.

So what's the end story? Well, I think this is a technology that we haven't seen the last of. The techniques are only going to get better and processors faster to let us do more number crunching. I'm curious to watch this develop and try to apply it in some basic experiments and see what happens. What would you try this technique on? Leave it in the comments!

Listening to Sound with a Bag of Chips

mit-visual-microphone

Today I wanted to share some thoughts on some great new technology that is being actively developed and uses cameras to extract sound and physical property information from everyday objects. I had heard of the visual microphone work that Abe Davis and his cohorts at MIT were working with, but they've gone a step further recently. Before we jump in, I wanted to thank Evan Bianco of Agile Geoscience for pointing me in this direction and asking about what implications this could have for rock mechanics. If you like the things I post, be sure to checkout Agile's blog!

Abe's initial work on the "visual microphone" consisted of using high speed cameras and trying to recover sound in a room by videoing an object like a plant or bag of chips and analyzing the tiny movements of the object. Remember sound is a propagating pressure wave and makes things vibrate, just not necessarily that much. I remember thinking "wow, that's really neat, but I don't have a high speed camera and can't afford one". Then Abe figured out a way to do this with normal cameras by utilizing the fact that most digital cameras have a rolling shutter.

Most video cameras around shoot video at 24 or 32 frames (still photos) each second (frames per second or fps). Modern motion pictures are produced at 24 fps. If we change the photos very slowly and we just see flashing pictures, but around the 16 fps mark our mind begins to merge the images into a movie. This is an effect of persistence of vision and it's a fascinating rabbit hole to crawl down. (For example, did you know that old time film projectors flashed the same frame more than once? The rapid flashing helped us not see flickers, but still let the film run at a slower speed. Checkout the engineer guy's video about this.) In the old days of film, the images were exposed all at once. Every point of light in the scene simultaneously flooded through the camera and exposed the film. That's not how digital cameras work at all. Most digital cameras use a rolling shutter. This means the image acquisition system scans across the CCD sensor row by row (much like your TV scans to show an image, called raster scanning) and records the light hitting that sensor. This means for things moving really fast, we get some strange recordings! The camera isn't designed to capture things that move significantly in a single frame. This is a type of non-traditional signal aliasing. Regular aliasing is why wagon wheels in westerns sometimes look like they are spinning backwards. Have a look at the short clip below that shows an aircraft propeller spinning. If that's real I want off the plane!

But how does a rolling shutter help here? Well 24 fps just isn't fast enough to recover audio. A lot of audio recordings sample the air pressure at the microphone about 44,200 times per second. Abe uses the fact that the scanning of the sensor gives him more temporal resolution than that rate at which entire frames are being captured. The details of the algorithm can get complex, but the idea is to watch an object vibrate and by measuring the vibration estimate the sound pressure and back-out the sound.

Sure we've all seen spies reflect a laser off glass in a building to listen to conversations in the room, but this is using objects in the room and simple video! Early days of the technology required ideal lighting and loud sounds. Here's Abe screaming the words to "Mary had a little lamb" at a bag of potato chips for a test.... not exactly the world of James Bond.

Screenshot 2015-09-12 13.19.56

 

As the team improved the algorithm, they filmed the bag in a natural setting through a glass pane and recovered music. Finally they were able to recover music from a pair of earbuds laying on a table. The audio quality isn't perfect, but good enough that Shazam recognized the song!

The tricks that the group has been up to lately though are what is the most fascinating. By videoing objects that are being minority perturbed by external forces, they are able to model the object's physical properties and predict its behavior. Be sure to watch the TED talk below to see it in action. If you're short on time skip into about the 12 minute mark, but really just watch it all.

By extracting the modulus/stiffness of objects and how they respond to forces, the models they create are pretty lifelike simulations of what would happen if you exerted a force on the real thing. The movements don't have to be big. Just tapping the table with a wire figure on it or videoing trees in a gentle breeze is all it needs. This technology could let us create stunning effects in movies and maybe even be implemented in the lab.

I've got some ideas on some low hanging fruit that could be tried and some more advanced ideas as well. I'm going to talk about that next time (sorry Evan)! I want to make sure we all have time to watch the TED video and that I can expand my list a little more. I'm thinking along the lines of laboratory sensing technology and civil engineering modeling on the cheap... What ideas do you have? Expect another post in the very near future!

Open Science Pt. 2 - Open Data

For the second installment in our summer open science series, I’d like to talk about open data. This could very well be one of the more debated topics; I certainly know it always gets my colleagues opinions exposed very quickly in one direction or the other. I’d like to think about why we would do this, methods and challenges of open data, and close with my personal viewpoint on the topic.

What is Open Data?

Open data simply means putting data that supports your scientific arguments in a publicly available location for anyone to download, replicate your analysis, or try new kinds of analysis. This is now easier than ever for us to do with a vast array of services that offer hosting of data, code, etc. The fact that every researcher will likely have a fast internet connection makes most arguments about file size invalid, with the exception of very large (100’s of gigabytes) files. The quote below is a good starting place for our discussion:

Numerous scientists have pointed out the irony that right at the historical moment when we have the technologies to permit worldwide availability and distributed process of scientific data, broadening collaboration and accelerating the pace and depth of discovery…..we are busy locking up that data and preventing the use of correspondingly advanced technologies on knowledge.

- John Wilbanks, VP Science, Creative Commons

Why/Why-Not Open Data?

When I say that I put my data “out-there” for mass consumption, I often get strange looks from others in the field. Sometimes it is due to not being familiar with the concept, but other times it comes with the line “are you crazy?”  Let’s take a look at why and why-not to set data free.

First, let’s state the facts about why open data is good. I don’t think there is much argument on these points, then we’ll go on to address more two-sided facets of the idea. It is clear that open data has the potential to increase the friendliness of a group of knowledge workers and the ability to increase our collaboration potential. Sharing our data enables us to pull from data that has been collected by others, and gain new insights from other’s analysis and comments on our data. This can reduce the reproduction of work and hopefully increase the numbers of checks done on a particular analysis. It also gives our supporters (tax payers for most of us) the best “bang for their buck.” The more places that the same data is used, the cost per bit of knowledge extracted from it is reduced. Finally, open data prevents researchers from taking their body of knowledge “to the grave” either literally or metaphorically. Too often a grad student leaves a lab group to go on in their career and all of their data, notes, results, etc that are not published go with them. Later students have to reproduce some of the work for comparison using scant clues in papers, or email the original student and ask for the data. After some rummaging, they are likely emailed a few scattered, poorly formatted spreadsheets with some random sampling of the data that is worse than no data at all. Open data means that quality data is posted and available for anyone, including future students and future versions of yourself!

Like every coin, there is another side to open data. This side is full of “challenges.” Some of these challenges even pass the polite term and are really just full-blown problems. The biggest criticism is wondering why someone would make the data that they worked very hard to collect out in the open, for free, to be utilized by anyone and for any purpose. Maybe you plan on mining the data more yourself and are afraid that someone else will do that first. Maybe the data is very costly to collect and there is great competition to have the “best set” of data. Whatever the motivation, this complaint is not going to go away. Generally my reply to these criticisms goes along the lines of data citation. Data is becoming a commodity in any field (marketing, biology, music, geology, etc). The best way to be sure that your data is properly obtained is to make it open with citation. This means that people will use your data, because they can find it, but provide proper credit. There are a number of ways to get your data assigned a digital object identifier (DOI), including services like datacite. If anything, this protects the data collector by providing a time-stamp of doing data collection of phenomena X at a certain time with a time-stamped data entry. I’m also very hopeful that future tenure committees will begin to recognize data as a useful output, not just papers. I’ve seen too many papers that were published as a “data dump.” I believe that we are technologically past that now, if we can get past "publish or perish," we can stop these publications and just let the data speak for itself.

Another common statement is “my data is too complicated/specialized to be used by anyone else, and I don’t want it getting mis-used.” I understand the sentiment behind this statement, but often hear it as “I don’t want to dedicate time to cleaning up my data, I’ll never look at it after I publish this paper anyway.” Taking the time to clean up data for it to be made publicly available is when you have a second chance to find problems, make notes about procedures and observations, and make it clear exactly what happened during your experiment (physical or computational). I cannot even count the number of times I’ve looked back at old data and found notes to myself in the comments that helped guide me through re-analysis. These notes saved hours of time and possibly a few mistakes along the way.

Data Licensing

Like everything from software to intellectual property, open-data requires a license to work. No license on data is almost worse that no data at all because the hands of whoever finds it are legally bound to do nothing with it. There is even a PLOS article about licensing scientific software that is a good read and largely applies to data.

What data licensing options are available to you are largely a function of the country you work in and you should talk with your funding agency. The country or funding agency may limit the options you have. For example, any US publicly funded research must be available after a presidential mandate that data be open where possible “as a public good to advance government efficiency, improve accountability, and fuel private sector innovation, scientific discovery, and economic growth.” You can read all about it in the White House U.S. Open Data Action Plan. So, depending on your funding source you may be violating policy by hoarding your data.

There is one exception to this: Some data are export controlled, meaning that the government restricts what can be put out in the open for national security purposes. Generally this pertains to projects that have applications in areas such as nuclear weapons, missile guidance, sonar, and other defense department topics. Even in these cases, it is often that certain parts of the data may still be released (and should be), but it is possible that some bits of data or code may be confidential. Releasing these is a good way to end up in trouble with your government, so be sure to check. This generally applies to nuclear and mechanical engineering projects and some astrophysical projects.

File Formats

A large challenge to open data is the file formats we use to store our data. Often times the scientist will use an instrument to collect their data that stores information in a manufacturer specific, proprietary format. It is analyzed with proprietary software and a screen-shot of the results included in the publication. Posting that raw data from the instrument does no good since others must have the licensed and closed-source software to even open it. In many cases, the users pay many thousands of dollars a year for a software “seat” that allows them to use the software. If they stop paying, the software stops working… they never really own it. This is a technique that the instrument companies use to ensure continued revenue. I understand the strategy from a business perspective and understand that development is expensive, but this is the wrong business model for a research company. Especially considering that the software is generally difficult to use and poorly written.

Why do we still deal in proprietary formats? Often it is because that is what the software we use has, as mentioned above. Other times it is because legacy formats die hard. Research groups that have a large base of data in an outdated format are hesitant to update the format because it involves a lot of data maintenance. That kind of work is slow, boring, and unfunded. It’s no wonder nobody wants to do it! This is partially the fault of the funding structure, and unmaintained data is useless data and does not fulfill the “open” idea. I’m not sure what the best way to change this idea in the community is, but it must change. Recent competitions to “rescue” data from older publications are a promising start. Another, darker, reason is that some researches want to make their data obscure. Sure, it is posted online, so they claim it is “open”, but the format is poorly explained or there is no meta-data. This is a rare case, but in competitive fields can be found. This is data hoarding in the ugliest form under the guise of being open.

There are several open formats that are available for almost any kind of data including plain text, markdown, netCDF, HDF5, and TDMS. I was at a meeting a few years ago where someone argued that all data should be archived as Excel files because “you’ll always be able to open those.” My jaw dropped. Excel is a closed, XML based, format that you must have a closed-source program to open. Yes, Open Office can open those files, but compatibility can be sketchy. Stick to a format that can handle large files (unlike Excel), supports complex multi-dimensional data (unlike Excel), and has many tools in many languages to read/write it (unlike Excel).

The final format/data maintenance task is a physical format concern. Storage media changes with time. We have transitioned from tapes, floppy disks, CDs, and ZIP disks to solid state storage and large external hard-drives. I’m sure some folks have their data on large floppy disks, but haven’t had a computer to read them in years. That data is lost as well. Keeping formats updated is another thankless and unfunded task. Even modern hard-drives must be backed up and replaced after a finite shelf life to ensure data continuity. Until the funding agencies realize this, the best we can do is write in a small budget line-item to update our storage and maintain a safe and useful archive of our data.

Meta-Data

The last item I want to talk about in this already long article is meta-data. Meta-data, as the name implies, are data about the data. Without the meta-data, most data are useless. Data must be accompanied by the experimental description, relevant parameters (who, when, where, why, how, etc), and information about what each data item means. Often this data lives in the pages of the laboratory notebooks of experimenters or on scraps of paper or whiteboards for modelers. Scanners with optical character recognition (OCR) can help solve that problem in many cases.

The other problems with meta-data are human problems. We think we’ll remember something, or we don’t have time to collect it. Anytime that I’ve thought I didn’t have time to write good notes, I payed by spending much more time after the fact figuring out what happened. Collecting meta-data is something we can’t ever do enough of and need to train ourselves to do. Again, it is a thankless and unfunded job… but just do it. I’ve even just turned on a video or audio recorder before and dictated what I’m doing. If you are running a complex analysis procedure, flip on a screen capture program and make a video of doing it to explain it to your future self and anyone else who is interested.

Meta-data is also a tricky beast because we never know what to record. Generally, record everything you think is relevant, then record everything else. In rock mechanics we generally record stress conditions, but never think to write down things like temperature and humidity in the lab. Well, we never think to until someone proves that humidity makes a difference in the results. Now all of our old data could be mined to verify/refute that hypothesis, except we don’t have the meta-data of humidity. While recording everything is impossible, it is wise to record everything that you can within a reasonable budget and time commitment. Consistency is key. Recording all of the parameters every time is necessary to be useful!

Final Thoughts

Whew! That is a lot of content. I think each item has a lot unsaid still, but this is where my thinking currently sits on the topic. I think my view is rather clear, but I want to know how we can make it better. How can we share in fair and useful ways? Everyone is imperfect at this, but that shouldn’t stop us from striving for the best results we can achieve! Next time we’ll briefly mention an unplanned segment on open-notebooks, then get on to open-source software. Until then, keep collecting, documenting, and sharing. Please comment with your thoughts/opinions!

Exploring Scientific Computing at SciPy 2014

Texas Campus

This past week I've been in Austin, TX attending SciPy 2014, the scientific Python conference.  I came in 2010 for the first time, but hadn't been able to make it back again until this year.  I love this conference because it gives me the chance to step away from work on my PhD and distractions of hobby projections to focus on keeping up with the world of scientific computing with Python.  I try to walk the fine line between being a researcher, engineer, and programmer everyday.  That means that it is easy to fall behind the state of the art in any one of those, and conferences like this are my way to getting a chance to learn from the best.

SciPy consists of tutorials, the conference, and sprints:

The first two days were tutorials in which I got to learn about using interactive widgets in iPython notebooks, reproducible science, image processing, and Bayesian analysis.  I see lots of things that I can apply in my research and teaching workflows!  Interactive notebooks are one of the last things that I was wishing for from the Mathematica notebooks.

The next three days were talks in which we got to see the newest software developments and creative applications of Python to scientific problems.  I, of course, gravitated to the geophysics oriented talks and even ran into some people with common connections.  It was during the conference that I gave my poster presentation.  I knew that the poster focused more on the application of Python to earthquake science than any earth-shaking (pun-intended) software development.  There were a few on the software side that wondered why I was there (as expected), but the poster was generally very well received.  Again I had several chance encounters with people from Google and other companies that had similar backgrounds or were just very interested in earthquakes!

The final two days (I'm writing this on the last day) were sprints.  These are large pushes to further develop the software while a critical mass of experts are in one location.  I'm still new enough to these massive open-source projections (on the development side at least) that I wasn't incredibly useful, but the reception of the developers was great!  Everyone was excited if you wanted to help and would spend as much time as needed to get you up and running.  During the sprints I've been following a fix for an issue that has recently caused problems in my plotting.  I also fixed a tiny issue (with help) and had my first pull request accepted.  For software people these are tiny steps, but for someone coming from just developing in-house purpose-designed tools.... it was an hit of the open-source collaboration drug.

Lastly, I worked on a project of my own during the evenings.  During the 2010 conference I worked with a friend to make a filter remove the annoying vuvuzela sound from the World Cup audio.  This year I've been making a fun earthquake visualization tool.  You'll be seeing it here on the blog, and may have already seen it if you follow me on twitter.  I learned a lot during this year's SciPy 2014, got to spend time with other alums of OU Meteorology, and meet some new folks.  Next on the blog we'll be back to some radar or maybe a quick earthquake discussion!

Reviving a Piece of the 1970's: ISEE-3

hello.again.m

There's been a decent buzz in the space and tech communities about the "ISEE-3 Reboot Project", so I thought it would be worth mentioning here and pointing out some of wonderful techniques they are using to revive a satellite from almost 40 years ago.

The ISEE-3 satellite is one of three satellites that made up the International Cometary Explorer (ICE) program.  There were some interesting orbital things done with this satellite after its launch in August of 1978.  It was also the first spacecraft to go through the tail of a comet!  As with all missions, this one came to an end and the satellite was not head from since 1998.  The equipment to talk to the satellite was removed and it was considered to be out of service.

ISEE-3 sits in a heliocentric orbit, meaning that orbits the sun, not the Earth.  We knew that ISEE-3 would make another stop by our planet in 2014 when it was parked in this orbit in 1986 (from what I can tell anyway).  A group of citizen scientists started the ISEE-3 Reboot project, crowd funded on the internet.  They got permission to take over the satellite and intend to use the Moon's gravity and a rocket burn to send it on another mission.  If the window of June is missed, the satellite will probably never be heard from again.

The team was able to contact ISEE-3 on May 29 using the Arecibo observatory radio telescope.   The craft was commanded to transmit engineering telemetry, basically a health screening of the systems.  The team is currently busy decoding the data (streaming in at 512 bits/sec) and planning how they will execute the rocket burn.

The team is running out of an old McDonalds at the NASA Ames Research Park, the makeshift mission control has been termed "McMoons" after hosting previous space based projects.

IMG_3113

 

The part of this that I find amazing is the role that software defined radio is playing.  Software defined radio (SDR) is a way to use software to emulate radio equipment.  With a small USB stick I can receive many different kinds of radio signals and decode them, something that would have required racks of equipment a few years ago.  This team is using a radio termed the "USRP" that allows them to transmit and receive.  I've written about them before (here) and have used them in research.  They are amazing little units and provide a unique learning opportunity.  (Maybe I'll post something about a radar we made with one of them as a demo!)  A photo tweeted by the team shows 2 USRP units and laptops hooked into the giant dish antenna at Arecibo.

Screen Shot 2014-06-02 at 12.51.08 PM

 

That's all for now, but stay tuned to the team's website for updates and I'll be keeping up with the progress as well.  This is just another incredible example of how advanced hardware and software that has become relatively cheap can allow a group of savvy citizens to accomplish incredible feats!  Way to go folks!

 

The Infrasound Bucket - Part 1 - Hardware

I'd like to write a short series of posts describing my setup of the infrasound unit I've written about before.  This is the same unit we used to look at traveling acoustic energy from the Russian meteorite and will soon use to examine earthquakes! Placing the unit inside my office or even inside the apartment proved to be very noisy as I saw every time someone opened or closed a door!  The makers (Infiltec) suggested that I put it outside, maybe in a drink cooler to shield it from the weather.  I did exactly that (photos below), but the cooler turned out to not be water proof and had about 2 cm water standing in the bottom when I checked it after a small storm.  The data quality while the instrument was outside was amazing though, with seismic signals coming through very clearly.  It was time to design a new system that would: 1) Be safe to leave outside in the weather, 2) Not have thick data cables running inside to a computer, 3) Would not require an inside computer, and 4) Would automatically post the current data online.

For the first post we're going to talk about the casing setup and mounting of all the vital hardware.  One weekend we decided to go wandering about Home Depot to find a suitable shell for the instrument as well as pickup a few other essential supplies.  Lendi had the flash of inspiration that we should use a 5-gallon plastic bucket... the ones at the Home Depot "Homer's All Purpose Bucket" even have an O-ring seal on the lid.  Perfect.

A built in O-ring seal on the bucket.

Now to figure out how to hold the hardware up off the bottom of the bucket.  In an ideal world this isn't needed, but in reality water may get in and I don't want it covering electronics thrown in the bottom of the bucket.  We used 1/4" plywood cut to a keystone shape that just fits the vertical profile of the bucket.  Adding two "L" brackets from the shelving section meant for ~$15 we had the shell and left over plywood.
Test fitting the plywood into the bucket.  Notice the cooler in the background that formerly housed the instrument.

I bolted the infrasound unit to the wood by using "plumber's tape" or metal strap with holes down its length.  This isn't the most elegant solution, but it meant no drilling the infrasound case which is semi-sealed on its own.  It is also very easy to get the unit out for any maintenance.   My RaspberryPi ended up having problems on the circuit board, so I've bolted a Beagle Bone Black to the board as well.

Front of the mounting board.  Infrasound unit (right), Beagle Bone (left), and power plugs (top left).

Rear of the mounting board with power passthrough.  

With no tall standoffs handy I made use of locking nuts, washers, and other assorted 4-40 hardware.

Two holes were drilled in the side of the bucket: one for the power and one for the air tube to the infrasound instrument.  I passed the power cable through (outdoor zip cord) through as well as clear plastic tubing and sealed it with bathroom silicon sealant.  I'd recommend sealing on the inside and outside of the bucket bulkhead.  Make sure to leave extra cable and tube for drip loops. A drip loop like structure was fashioned on the outside of the bucket to ensure no rain would blow up the tube into the unit.  We taped the tube down and then ran beads of silicon to secure it to the bucket.  After the sealant dried we moved the tape and secured the rest of the tubing.

Power and air tube sealed into the bucket and loops to prevent water flow.

Inside the bucket: notice the power plug.

In later posts we'll talk about how the power is actually provided and such, but the part that pertains to the hardware is the mounting of two binding posts on the plywood at the standard 3/4" spacing.  This allows us to power the board from a banana jack on the bench for testing or operationally in the bucket.  I drilled a passthrough hole to send power from the back of the jacks to the front of the panel.

Initially I built a 5V regulator to power the computer with from an LM7805 linear voltage regulator, but this was indeed a poor choice.  Even with a decent heat sink, the chip still got blistering hot when I was drawing 700mA (of the 1000mA rated power).  Considering this would be outside in the summer heat and the fact that I didn't want the failure point of a mechanical fan I decided to use a buck voltage converter.  Linear regulators dissipate all extra power as heat.  For example: I was feeding 12VDC to the converter with a 700mA load running at 5VDC.  That means that (12V-5V)*0.7 = 4.9 Watts of power was being turned into waste heat! No wonder, remember we think of watts as energy/time (Joules/second actually).  That's a lot of wasted electricity and really just a complication to our design, but it was very clean power.

The old linear regulator.  It's now awaiting a new use in the parts bin.

The buck converter is a switching type regulator.  I don't want to get into how switching regulators work current, but it's an interesting topic and you should have a read on the theory if you like.  I bought a small unit (P/N 1385) from Adafruit that is rated to 3A (though it gets warm there).  The power isn't quite as clean from this switching supply, but it's fine for out use here.  It works great with the Beagle Bone and provides lots of extra power for 5V accessories.  Don't want to order and ship from Adafruit? You can get the exact same thing from a model shop.  They are called "battery eliminator circuits" and allow modelers to plug their airplane, car, etc servo electronics (5VDC operation) into a 12V battery they already have in their kit.  Just clip the 3 pin servo plug off the end and you are ready to go.  Don't forget good soldering practice and to use heatshrink tube! Shorts could spark a fire, which we don't want.

The "battery eliminator circuit" or my 5V buck converter to supply 5VDC to the Beagle Bone.

So there it is! Next time I'm going to talk about setting up the power and network infrastructure.  Maybe even the serial communications! We're going to try to avoid using a serial-USB converter since the Bagle Bone has only one USB port (that I'm using for a WiFi adapter), I don't want to use a hub, and it's a chance to learn about signal level shifting and wire into that temping header on the board.

Everything fit into the bucket nicely and powers up from the bench power supply.

The iPad: How it is Revolutionizing Field Work

It's not even been one year since the iPad hit the market and it is well on it's way to becoming an essential for many people around the world.  According to CNN, the iPad has the fastest adoption rate of any consumer advice (read the full article here).  I know that the iPad is difficult to put down, after standing in line all afternoon the day of the 3G release I was entertained the entire weekend.

But, what else can you do with the iPad.  We've all seen the movies, games, and flashy organization apps in the adds, but what about more difficult work?  The productivity category was initially slow to start, but now is full of options.

The numbers/pages/keynote set is $30 ($10 each) and has saved me several times.  When I needed to make a promotion slide last minute at a conference keynote came to my rescue.  I simply took images I needed from various emails and online, added some text, and in 10 minutes had a decent looking slide to submit.  Numbers has allowed me to use some handy computational workbooks in the field to make very simple models of data that is coming in.  Last, pages is very handy for quick edits on the road, or when out somewhere on campus.  There are a few glitches, but they have continually be improving, especially in the area of importing Microsoft Office documents.  So far, no track changes exists, but hopefully that will be coming soon.

For quick remote server administration I use iSSH.  This is really a fantastic app with the exception that the arrow keys/command keys on the bluetooth keyboard don't work forcing you to use onscreen keys.  This is the only limitation that prevents me from doing some full scale programming while connected to another machine back at home.

It's always important to know the weather while in the field and I use a combination of Storm Spotter and Radar Scope.  The developer of Storm Spotter is another OU meteorology student and I highly recommend his app.  Radar Scope does have a few things like spotter network, but it does not have any form of surface street map.  Storm Spotter uses google maps which makes exact location or storm chasing much easier.

Another field essential is taking notes.  There are many note apps out there and most do about the same things with different degrees of reliability.  For quick sketches I use Penultimate and for class notes I use Note Taker HD, which has a 'zoom box' that lets me write large with my stylus (the Pogo Sketch) and it is normal sized writing on the page.  Sundry notes is also around, but has not received any use by me for some time.

File sync is also an essential and can be done with Dropbox.  I already loved this service and the mobile app made life easier! Now I can save notes from the field and they instantly sync to my computer at home, my phone, and my laptop.

For field math there is Wolfram alpha (cloud service), Quick Graph, and many apps like MathTasks that do simple calculations on the fly.

Sometimes I'll use a GPS utility to mark out rough locations on a map or even the iPad ArcGIS to get an approximate distance/area.

Finally, we all need files and file editing in the field.  I use Papers to keep my scientific paper library with me at all times.  In the field or at a conference it's easy to find that paper you need a snippet from and email it directly to the interested party/conference goer.  Annotating PDF files is easy with iAnnotate PDF and viewing large files is nice with GoodReader (though Books now does this also).

While all these apps are productivity, you can bet all iPad owners have their favorite music service loaded, Netflix, and other entertainment too.  While I do love using my iPad it does have overheating issues when working out in direct sun on a hot day.  The screen is great at letting solar radiation in, and trapping the re-emitted IR inside the device,  shutting it off in ~10 minutes.  What can you do? Use a case with an open back and keep the screen shaded.  I'm not quite ready to quit carrying paper notes all together, but it's getting close and my daily/travel backpack is getting lighter every year.  Read about other great apps for large scale studies used in Pompeii here.

Images are property Apple.

The Vuvuzela: An Annoying Horn but a Fun Filter Project!

As we've watched the world cup matches over the past weeks everyone has been annoyed by the droning hum of the vuvuzela.  Everyone in the crowd blowing on one of these pipes makes life for our ears unpleasant.  What can you do though?

While at SciPy 2010 this week we saw a tutorial about signals with Python.  Another student and I talked about filtering out the drone and after several late nights of hacking it worked! The principle is simple, block the frequency of the horn and it's harmonics.  To do this we use a notch filter that rejects the signal from a certain frequency range.

The code (waves.py) is available and is a short script to read in, filter, display, and write out the files.  Below I show an example of the signal and power spectral density before filtering (left) and after filtering (right).  There are also links to the audio files.  I found three example files and filtered them with three different filter widths (5,10,25 Hz). All runs are available at http://leemanwebb.org/vuvu/ , but that link will disappear and I'll then post a more permanent page after some tweaking with the project.  Be sure to listen to example 3 before and after the 5Hz width filter.