editors note: this is a 2018 update to an article Bob wrote for Make magazine, volume 27, July 2011
ImageJ is a freely-available program for processing images and videos. It was made for scientists for images made on microscopes, but it’s available to anyone. And anyone is free to add new functions to the program. As a result, it improves constantly by contributions from the more programming-savvy among its users. In this sense, ImageJ is a great resource for images and videos much as the open source browser Firefox is for web browsing, or indeed as Wikipedia is for information: each will work on just about any computer, and anyone can make them better. But ImageJ is not yet well known outside of science.
As a scientist and a fan of creative tinkering, I thought it would fun to see what might result if Make readers were introduced to some of the tricks that ImageJ can perform. If you have a few images or a short video made on a digital camera or a webcam, you can expect to transform them as in the examples that follow within about 20 minutes.
Where ImageJ came from
ImageJ was written for biomedical researchers by an employee of the US National Institutes of Health, programmer Wayne Rasband. It’s Java-based, so it works on PCs, Macs, and Linux computers. Rasband designed ImageJ with an open architecture: anyone could write plugins to add new tricks to the program.
ImageJ was first released way back in 1997, in the ancient, pre-Google era of computers. Rather than losing popularity over the years, its open architecture is making it an increasingly valuable tool to science. I was curious to know if Rasband foresaw the potential of an open architecture way back in 1997, so I emailed him. “I always liked seeing other people use the software I wrote”, wrote Rasband, “and I have always made the source code freely available. I discovered if I created software that was easy to extend, and I gave away the source code, then I would get code contributions back from the user community.”
What it can do
ImageJ is useful for processing images using filters similar to those found on programs like Photoshop. With hundreds of plugins available, there are hundreds of tricks to try. My own favorite tricks are the ones that can transform a video you’ve made by displaying motion from the video on a single image. I also like using ImageJ to perform math on images — combining two images by simply adding pixel intensities, for example.
Get started
You’ll want to start by downloading the ImageJ application and adding a couple of useful plug-ins:
- Download ImageJ
Start by downloading ImageJ from the Fiji site. Fiji is just a version of ImageJ with a set of plugins already included and with an automatic update function so that your copy of ImageJ will keep up to date with useful changes. If you are using a Mac, put Fiji in your Applications folder. - Add some extra plug-ins
There are loads of plugins that you can add, with bewildering names like Point Picker, Spectrum Extractor, and Lipschitz Filter.For now, I recommend adding just one useful set of plugins, some of which you’ll use in the examples on the next few pages. This set is called “Cookbook” and can be uploaded easily using these instructions.The Cookbook plugins will appear under a new pulldown (at the top of your screen) called “Cookbook”.
Follow along and try the examples below with your own images
On the next few pages, I show some images that I made using ImageJ, along with a short description of each image so that you can get an idea of how each image was made.
In italics below each example, you can find a detailed, step-by-step “what to click” protocol for each of these examples, so that you can try doing similar things with your own photos and videos. You may want to try the examples at first with small images and very short videos, and perhaps using grayscale images and videos (or converting them to grayscale or 8-bit RGB when prompted), since the larger your files, the longer each step will take.
1. Image calculator
The Image Calculator tool can combine images in various ways using simple math, for example by adding, subtracting, or averaging the colors of each pixel. To illustrate, in the images shown above, the pixel in the lower left hand corner of the snowflake image has a color that’s coded as 12,6,10 in RGB, meaning that it has Red set at 12, Green set at 6, and Blue set at 10. These numbers are out of 255, with 0 being dark, and 255 bright. So a pixel color of 12,6,10 — all low numbers out of 255 — means that it’s a pretty dark pixel. The pixel at the corresponding position in the Lincoln photo is colored 70,54,42. The result of averaging these two? A pixel with the color 41,30,26: exactly the average between the two sets of numbers. The Image Calculator treats each pixel this way.
This can be a powerful tool for combining images in predictable ways. What would happen if you displayed the difference between two nearly-identical photos?
You’ll see when trying the Image Calculator that adding, subtracting, and averaging are just a few among several ways to combine images.
Click FILE / OPEN and select an image on your computer, and then repeat to open a second image. Then click PROCESS / IMAGE CALCULATOR, select each of your image names, and then select an operation like ADD, AVERAGE or DIFFERENCE to combine the images.
2. Projecting a stack
Here, I’ve taken a short video during a lightning storm. This lightning flash took about a second to cross the sky. On the video, distinct parts of the flash could be seen in separate frames of the video (the 10 frames at top in the image below). To see what the whole flash looked like, I added the light in frame 1 to the light in frame 2, frame 3 and so on. To do this, I opened the video in ImageJ and used a tool called Z Project to make a single image.
The camera was left perfectly still during the brief video, sitting on a chair instead of in my moving hand. This turned out to be important: the images aligned well in the final projected image, but they would not have if the camera was moving.
You’ll see a number of ways to open files of different formats under FILE / IMPORT, but there’s an easier way: most files can be opened by just dragging and dropping them onto the row of tool icons. Before you open a video file, though, you may want to use a video editing program to trim your video to a short length. ImageJ can handle a video that has a few hundred images, but cramming in longer videos can slow or crash the program. As you open your video, you’ll be offered an option to convert it to 8-bit grayscale or RGB, and to use “virtual stack” rather than loading your whole video into memory. Doing these things will make steps that follow go more quickly.
iPhones unfortunately save movies in a format that ImageJ can’t use, so you’ll need to convert those to sequences of individual photos first using a tool like the Video 2 Photo iPhone app, or Quicktime Pro. Then you can open your folder of photos by clicking FILE / IMPORT / IMAGE SEQUENCE.
Once your video is opened in ImageJ, click on it to select it. You can scroll through it using the < and > keys. Then click IMAGE / STACKS / Z PROJECT and choose MAX INTENSITY to sum the brightest pixels from each frame onto a single image.
The montage at the top of this example was made with an ImageJ tool too. Click on your video again and then click IMAGE / STACKS / MAKE MONTAGE to make a montage.
3. Projecting a stack with a dark subject
This image was made in the similar way as the last one except here the subject was darker than the background. The video used as a source here was taken from a camera left still, pointing at the sky as birds flew over. Then a short 2-second segment of the video, about 30 frames long, was projected in ImageJ.
Open your video by either dragging a video onto the ImageJ/Fiji toolbar or by opening an image sequence using FILE / IMPORT / IMAGE SEQUENCE. Then click on your video window to select it, click IMAGE / STACKS / Z PROJECT and choose MIN INTENSITY (instead of MAX this time) to add up the darkest pixels from each frame.
4. Highlighting objects that move
We made a time lapse video of Mexican jumping beans moving, and then tried to highlight the moving beans’ paths. A few ways of displaying the paths are shown in the figure below. The upper right image shows the same trick we used in the preceding example. In the lower left is the same trick but choosing “STANDARD DEVIATION” in place of “MIN INTENSITY”. This trick results in moving objects that pause for a long time appearing the brightest. Combining the resulting images using Image Calculator (as in example 1 above) can also give interesting effects.
Below, I describe how to simply subtract from each frame everything that was also on the previous frame, so that pixels that don’t change over time appear black, and those that do change over time appear in lighter shades (the lower right image).
Close any windows you have open in Fiji. Then open your video by either dragging a video onto the ImageJ/Fiji toolbar or by opening an image sequence using FILE / IMPORT / IMAGE SEQUENCE. When the option pops up to “convert to 8-bit grayscale”, click the box to accept this. I recommend not choosing the virtual stack option this time. Then click on the video to select it, and make a second video showing only the moving objects by clicking COOKBOOK / T-FUNCTIONS / DELTA F UP (“Cookbook” is the set of plugins that you installed into Fiji by following the instructions in the “Get started” section above).
5. Color coding time
My kids and I made a time-lapse film of the stars passing over our yard from dusk to dawn. (To make the film, we used a Canon point and shoot camera hacked with open source CHDK software to do long-exposure time-lapse recording. The NightCap iPhone app is also okay at doing this using a smartphone). Then we used ImageJ to make a black and white film of only the moving objects. Then the film was time-coded with color. Lastly, all of the colored frames were projected on top of each other onto a single image. Color equals time in the final image: purple is just after sunset, and orange/yellow is later, just before sunrise. For example, the clouds appeared just before sunrise, so they appear orange. The stars were going by all night long, so they’re in multiple colors.
This example used multiple plugins, but it really took only a few clicks in ImageJ. Open your video by either dragging a video onto the ImageJ/Fiji toolbar or by opening an image sequence using FILE / IMPORT / IMAGE SEQUENCE. When the option pops up to “convert to 8-bit grayscale”, click the box to accept this. Then click on the video to select it, and make a second video showing only the moving objects by clicking COOKBOOK / T-FUNCTIONS / DELTA F UP. Then, to color code time, click COOKBOOK / Z-FUNCTIONS / Z CODE STACK. You can select from a number of color schemes. Now click IMAGE / STACKS / Z PROJECT… and choose MAX INTENSITY to add up the brightest pixels from each frame.
6. Kymographs
My sons and I found a robin’s nest on our house, and we were fascinated to have a peek. So we set up a webcam and watched the nest. On one day, we made an all-day time-lapse recording. The mother sat on the eggs throughout the day, leaving periodically for food. We were curious if there were any patterns to the timing of her trips, so we made a kymograph — a graph displaying specific positions over time. Here, the positions chosen were under a thin line that crossed the eggs.
In the kymograph at bottom, time is marked along the top, running from morning at the left to evening at the right. When the blue of the eggs is visible in the vertical stripes, the mother bird was out of the nest. We had read online that robins never leave their nests for more than 10-15 minutes at a time, but it looks as if she took a long lunch around 11:40-noon. It got dark just before 7pm.
Open your video by either dragging a video onto the ImageJ/Fiji toolbar or by opening an image sequence using FILE / IMPORT / IMAGE SEQUENCE. Then select the straight line tool from the row of tool icons, and click and drag to draw a line over an area of interest in the video. Click IMAGE / STACKS / RESLICE to see what happened under that line over time. If you want to sample more than just a thin line, then instead of using the straight line tool, select the rectangle tool from the row of tool icons, and click and drag to draw a rectangle over an area of interest in the video. Click IMAGE / STACKS / RESLICE, and then click IMAGE / STACK / Z-PROJECT. Your kymograph will appear vertically, with time running from top to bottom.
7. Take your pixels out of ImageJ and treat them to your own math
You can extract the pixel values from your image and try transforming them yourself if you’re handy with a spreadsheet program like Microsoft Excel. It’s interesting to see how features like local contrast can be highlighted this way. To illustrate this, I’ve used a 50×50 pixel image of an acorn below (A), generating several versions of the original image using simple formulas in Excel. B is a grayscale version of A. In C, the image has been blurred by averaging each pixel’s value with that of its neighbors. In D, areas of high contrast are highlighted after comparing each pixel’s value with its neighbors. In E, just the areas of high contrast are shown. In F, the degree of contrast in each area is converted to continuous grayscale. And in G, the grayscales in F were converted to colors using a lookup table in ImageJ.
I recommend that you try this on a tiny image, about 50×50 pixels, because some steps below will work only very slowly on larger images in Excel or other spreadsheet programs. Then convert your image to grayscale by clicking IMAGE / TYPE / 8_BIT. Then save the image using FILE / SAVE AS / TEXT IMAGE. This produces a text file that you can then open using a spreadsheet program like Microsoft Excel.
In the spreadsheet, you’ll see an array of numbers, each representing the pixel value of an individual pixel in a grayscale version of your image. I’ve used a 50×50 pixel image of an acorn (A) in the example below. I opened it in Excel as described above, tried out some things to transform that array of numbers, and then to see what image the new numbers would produce, I copied those cells back into a text editor (I used TextWrangler for Mac, but loads of others are available). I saved that file with a filename followed by .txt, and opened that file in ImageJ by clicking FILE / IMPORT / TEXT IMAGE. A grayscale image of the acorn resulted (B). Then I went back to the Excel file and tried playing around with the set of numbers. An Excel file containing each of the transformations in C-F is available here. If you’d like to color a grayscale image, you can open it in ImageJ, and click IMAGE / LOOKUP TABLES and pick a color scheme.
Figure 7: Image of an acorn transformed in several different ways
Experiment and have some fun
Once you’ve downloaded the program, it can be interesting to open a photo or import a video that you’ve taken and start clicking away to see what various buttons do. For those who prefer a more systematic start, there’s are guides available here or here.
In case you ever want to try more plugins, here’s a link to a big set of plugins. One plugin that I especially like is “Running Z projector”.
To add any of these plugins, first download it to your computer. If you’re using a Mac and it tells you the files can’t be opened because they’re from an unknown developer, you can bypass this by opening your Mac’s System Preferences, then open Security and Privacy, and you’ll see an “Open Anyway” button. Then open Fiji, click PLUGINS, then INSTALL… and select the file you downloaded.
Contribute to science yourself!
Makers and scientists both form creative communities, and communities that could probably learn a thing or two from each other. If you can program, and you see an interesting way to display images that no existing plugins can yet do, then why not write a new plug-in? Hundreds of plugins exist already, but great, new plugins appear every year, and the best plugins probably have yet to be written. Here’s an overview on creating new plugins.
Who’ll make use of your plugin, and what scientific discoveries might it help propel? Biomedical research articles are increasingly found in full form online, so in the months to years after submitting a plug-in, search online for the name of your plugin to find out how it’s contributing to science.
ADVERTISEMENT
