Air Quality Sensors: How Good is Good Enough?

Air quality sensors are being developed, sold, and deployed by more and more people, not just by professional air quality scientists like myself. This is a truly exciting time when citizens can help inform science with low-cost sensors and instruments, as long as the quality of the measurements is good enough.

People constantly ask me: What’s the quality of this sensor? How good is it? Does it really work? They ask me because I work at Sonoma Technology, Inc. (STI), which provides high-end consulting to air quality agencies like the U.S. Environmental Protection Agency. At STI, we use expensive, extremely accurate instruments to assess air quality conditions for scientific studies that affect policy and regulatory decisions. But at home, during nights and weekends, I’m hacking and working with low-cost air quality monitors with the DIY, non-profit, and maker communities.

Everybody wants a short, pithy answer to the quality question. Something like, “It’s accuracy is 98%” or “The sensor is as good as a $20,000 air quality instrument.” Unfortunately, there are no simple answers like that. This frustrates people, and it frustrated me when I first started working with air sensors in 2010. The best answer for the accuracy question is, “It depends on the application and how you plan to use the data. Is the project an educational one, a screening activity, a science study, a litigation case, or something else?” All of these projects require measurements with varying degrees of accuracy and quality: for some decisions, the stakes are high and the data had better be solid.

Instead of getting too frustrated, I decided to conduct my own DIY data quality evaluations in my garage and backyard. I did this for several reasons: 1) it would help me understand what’s doable, 2) sensor manufacturers typically do not have or publish real-world evaluation data, and 3) other organizations were not evaluating these emerging low-cost air sensors.¹ So I decided to focus my efforts on one of the most insidious air quality pollutants: particulate matter that is less than 2.5 micrometers in diameter. Also called PM_2.5, these particles cause lung and heart health problems in the U.S. and worldwide (epa.gov/pm/pdfs/pm-color.pdf, cdc.gov/air/particulate_matter.html).

¹ The U.S. EPA recently started evaluating the quality of lower cost air sensors using their sophisticated, controlled laboratories in Research Triangle Park, N.C. [Long, R., Beaver, M., Williams, R.  Kronmiller, K., Garvey, S.  Sensor evaluation considerations (procedures and concepts) of EPA’s ongoing evaluation efforts. Environmental Manager (August 2014), In Press.]

I conducted three studies to compare the quality of low-cost instruments to reference instruments accepted by the air quality community as providing quality PM_2.5 measurements. For each study, I evaluated one or more PM_2.5 sensors by measuring the ambient (real-world) concentration of particles or by generating particles from burning different types of fuel (wood, incense, kerosene, paper). Several types of analyses helped establish the quality of the new sensors: a scatterplot showing the relationship between the new sensor and the reference; a correlation analysis that shows the level of agreement between sensors; and a simple plot of the data to make sure they make sense conceptually.

Experiment 1: Winter Woodsmoke

Reference: Thermo PDR-1500

Sensor: Dylos (DC1100-PRO-PC)

Method: During the winter of 2012, I ran a Dylos sensor in my backyard in Santa Rosa, Calif., to measure the minute-by-minute fluctuations in particles due to residential wood burning in fireplaces (burning mostly for ambiance, not for home heating). I compared the results to the PM_2.5 measurements from the Thermo reference instrument.

Results: The scatterplot shows a very good linear relationship between the Thermo reference instrument and the low-cost Dylos sensor. A correlation of 0.92 indicated that 92% of the variations in the reference PM_2.5 data are also reflected by the Dylos count of particles. This agreement allows anyone to run a Dylos and relate the particle count measurements to PM_2.5 concentrations. In addition, as shown in the second Figure, the Dylos makes sense conceptually, because the daily changes in PM_2.5 are related to time of day and changes in the weather conditions. At night, people begin burning wood, and the resulting particles are trapped near the Earth’s cool surface; thus, PM_2.5 concentrations increase. During the day, PM_2.5 levels decrease as the air and particles are mixed vertically in the atmosphere by the warmer temperatures and stronger winds. It is encouraging to see the Dylos mimicking the patterns shown by the reference instrument.

Conclusion: This agreement is quite remarkable and encouraging. How good is the Dylos? Good enough to detect changes in particle levels due to wood smoke in Tim’s backyard.

More info at https://sites.google.com/site/airsensors2013/final-materials; see Posters, page 27.

Figure for Experiment 1:

Experiment 2: Does Particle Color Affect Sensor Response?

Reference: Thermo PDR-1500

Sensor: Dylos (DC1100-PRO-PC)

Method: The Dylos sensor detects particles when a particle flowing through the device scatters light from an LED into a photo detector. The color, size, and shape of the particles are affected by the type of fuel being burned. I was interested in how the Dylos sensor would respond to different types of particles. My garage served as a mini smoke chamber to burn different fuels, create smoke, and then collect data from the reference instrument and sensor.

Results: The scatterplot compares the Dylos to the reference monitor for all the fuels that I burned. Plant-based materials (wood, paper, tobacco) produce whiter particles, while petroleum-based fuels produce darker particles. Whitish particles scatter more light; dark particles absorb more light and are harder to detect with this sensor. The response curves are shown as solid lines. Ideal response curves are ones where a change in the reference instrument corresponds to a similar change in the sensor reading. Notice the very good response to wood and paper (higher light scatter); however, the response to kerosene is not as good (darker and less light scattering).

Conclusion: Agreement is good for whitish particles (from plant-based fuels), but not as good for darker, sootier particles from petroleum-based fuels.

Figures for Experiment 2:

Experiment 3: Particulate Matter Sensors for High School Students

Reference: Thermo PDR-1500

Sensor: Shinyei PPD-62PV

Method: STI is working with HabitatMap to develop a sensing device for high school students to measure particles and interpret the data they collect (KidsMakingSense.org). I tested this smaller PM_2.5 sensor with two approaches: 1) evaluate ambient particle concentrations at night, when neighbors burned wood in their fireplaces and smoke drifted into my backyard, and 2) evaluate particle concentrations when I burned wood in my backyard grill to purposely create higher concentrations of wood smoke particles.

Results: Both tests showed that this lower-cost sensor compared well with the reference instrument. The agreement between the Shinyei sensor on the scatterplot is reasonable (linear with a correlation greater than 0.7). In addition, the time-series plot of the data shows that variations in the particle count of the sensor correspond to changes in the reference measurements of PM_2.5.

Summary: The Shinyei sensor is sufficient to detect low, medium, and high particle concentrations. They are also sufficient for students to detect particles in the classroom, their homes, and surrounding neighborhoods.

Figures for Experiment 3:

Summary

Other people are starting to conduct similar evaluations for particles and gaseous pollutants. And I suspect that, in a year or two, we’ll have a wide range of evaluations to characterize what types of sensors are of sufficient quality for a particular application. So, how good are these sensors? Well, good enough to detect particles from wood smoke.