The Floor Chart is a 30×30 grid of buttons and matching 30×30 grid of LEDs. Each button is underneath an LED. The floor chart code can be thought of at a high level as taking input from the buttons and translating those into color changes in the LEDs. The ultimate goal is that clicking a particular button results in the cycling through a set of colors for the corresponding LED.

The 30×30 grid of buttons is made of 30 vertical strips overlaid on 30 horizontal strips. When pressing a particular place on the grid, one vertical strip and one horizontal strip will connect completing a circuit. This connection can be read as inputs to the digital pins on the arduino. Furthermore, the leds can be controlled through digital pins on the arduino turning them on and off and changing the colors. Each arduino has about 50 of these digital pins. Because of this limited number of digital pins, it would not be possible to hook up all 60 metal strips and the 30 LED strips. For this reason the 60 metal strips are broken up in half. So one arduino reads the input from 30 horizontal strips and 15 vertical strips, this means one arduino can read 450 of the 900 total buttons. So there are two arduinos reading in inputs from buttons.

Once these two arduinos get the button input, they need a way to relay that information to the corresponding LEDs. The way they do this is by sending that information to a third arduino that then relays that information to the LED on one of the 30 LED strips that matches up with the button that was pressed.

The Details

The code to do what was described above is broken down into three parts. The code that controls changing the color of the LEDs (the Master code), the code responsible for reading button presses in two separate arduinos (Minion code), and the code that supports reading button presses by reading changes in the current of the strips.

master.ino

The Master code is run on the arduino that changes the color of the LEDs. The Master code is called Master because it is responsible for all communication between arduinos as well as changing the color of appropriate LEDs. In order to change the color of specific LEDs the Master must first get information about button presses. It does so by setting up an I2C connection with the two Minion arduinos (arduinos that read button input) and polling each of the minions at intervals to see if they have received any button input. If the minion has observed any button input it will tell the Master by sending a payload over I2C that contains the address of the Minion, the row and column of the metal strips that changed state as well as the type of state change such as pressed or released (discussed further in Minion section). If there is no button input, the Minion will send an empty payload.

The row and column of the button as well as the information about the type of state change it is experiencing can be used to then update the colors of the LEDs. In order to update these LEDs we must have a way to keep track of the state of every pixel. The state of every LED is stored in an array with a slot for every pixel. Each element of the array is an unsigned integer of 32  bits. The most significant 24 bits of this integer store the time of the last interaction with the LED (this supports long presses and other time sensitive button inputs) while the least significant 8 bits store the state of the pixel.

When receiving a button input the Master will look up the current state of that pixel and take the appropriate action dependent on the current state and the input. For example if the pixel is currently green (ex: state 2) and its button gets pressed, it might be the case that the pixel should change to red (ex: state 3). In that case the Master would look up the state of the pixel which would be 2 and change the pixel to red based on that state and the fact that there was a button press. Once the pixel is changed physically, the state is then updated in the pixel state array.

This process of polling the Minions for button input then updating the state of the LEDs both physically and in their state array is repeated on an interval, in this case every 100ms.

masterTest.ino

This code is used to test that all of the LEDs are connected correctly. This code will light up each of the LEDs one at a time, first red then blue and repeat.

minion1.ino/minion2.ino

As stated above, the Minion code is responsible for reading button changes and sending those changes to the master. Each minion takes on half of the button grid to read for changes. The way the Minion reads information from its digital pins is by creating a Keypad which is a set of code that represents the vertical and horizontal strips of metal that make up the button grid and allows the state of each button to be read and repeatedly requesting to update the state of the Keypad.

When starting, each Minion joins the I2C bus with the master and sets up a function to be called when the Master wants information about the state change of buttons. This function grabs a button change from the Keypad containing the row, column, and type of button change. If there was not button change since the Minion last checked, Keypad will return a button change with all 0s for values. This button change is then packed up into a payload with the address of this Minion (used in the Master to adjust rows and columns depending on which half of the inputs the Minion reads), the row and column of the button change, and the type of button state change (press or release).

The Minion continually requests that the Keypad update the state of the input grid and only stops to send button state changes when requested by the Master.

Keypad.cpp/Keypad.h

Keypad is a data structure that supports the Minions in reading the state of the input grid (buttons). Keypad stores the row and column digital pin assignments as well as the overall size of the grid. Keypad also contains a method getKeys() for the Minions to call that updates the state of each of the buttons.

The way the update method works is that it can be called every 10ms max. If it has been at least 10ms since the last time called, it will scan they grid to check for any vertical strips touching any horizontal strips. This is done by sending a current through each vertical stirp one a time and reading the state of each of the horizontal strips to see if they are connecting with the vertical strip that has current running through it. This way the current state of the grid can be found.

Once the current state of the grid has been updated the code compares the current state to the previous state and if there is any change it adds that change to a buffer. The buffer is used as the Master can request button updates slower than the Minion can read button inputs. It could then be the case that the Minion finds two button changes in the time it takes the Master to request a button change. If there was no buffer the Master would only get one of the button changes. Once the button change is added to the buffer the previous state is set to the current state and the process of updating the state of the buttons can start again.

When the Minion requests a button change event, Keypad simply pops the first one off the buffer it has. This change event is the same one that the Minion then sends to the Master.

Overall

Overall the Minions keep updating the current state of the grid and find the button changes. The Master then communicates with the Minions to find out the button changes. Once it has a button change the Master then updates the state of the LEDs as appropriate. This process can be seen as drawn below.

We currently have three students working on the project. We had a fourth last spring, but she has since graduated. For the rest of us left, we now have specialized tasks to help move the project forward on all sides. Here the specialized tasks:

• Courtney Allen (HCI) works on the physical components of the circuit.
• Noah Sutter (CS) works on the programming for the buttons and LEDs.
• Sarah Schoemann (DM) works on the physical fabrication of the quilt.

When we last left the project, the LEDs were failing to stay on and communicate with the master microcontroller. After some diligent testing to make sure the LED strips weren’t blown, we discovered that the LEDs were actually not receiving enough power. It was at this stage when the specialized tasks came into play. Courtney investigated similarly large-scaled LED projects to discover that not only was there not enough power but the wires being used weren’t capable of the traversing power needed AKA we were on track to start something on fire. Since the discovery, the team has reached out to another group working successfully on a large-scale LED project on campus. With the other group, our main contact is with Noah P. (a different Noah than the one on the team) and he helped set up our next steps, which included scaling down our project.

That’s where the project finished off in the fall. Courtney converted the large 30×30 grid of LEDs into a 5×5 grid of LEDs; Noah adjusted the code to fit the new layout while keeping the code scalable for the next steps. After some testing and checking of the wired connections and the direct, now appropriately amped power supply, the interaction was functional for the first time since a year and a half ago and the very first time for the current group of students on the project! Getting ambitious, Noah extended the code to create a 5×30 grid, where 30 pixels on each of the 5 strands were connected and correctly responding to the button presses. The code will be broken down further in another post.

Here’s the project working on the 5×30 grid:

The LED strips turned on by way of pressing the capacitive “quilt” underneath (the dark piece underneath the strips).

It’s working!

Link to video of 5×30 Grid Working!

So, the next steps are to create a modular layout of the Neopixel strips. We have 30 total to power, but our power source can only power 5 strips at a time. As such, we are dividing up the 30 strips into 6 sections. One tricky part of creating a modular setup is to make sure all of the data pins from the Neopixels have a resistor and have long enough (but not too long) wire to connect; if the wire is too long, the resistant from the length will cause communication issues and then the strips won’t light up correctly. Another tricky part of the aimed modular setup requires the system to be easily dismantled and easily reassembled, which directly affects the connections and types of wires being used within the system. Currently, the wires work for the set of 5 strips, however the system is not stable and does not hold very well to movement.

Here’s how the testing setup is laid out:

The temporary scaled-down setup for 5 strips of LEDs.

As you can see, the ground is also connected to the Arduino, though the strips and the microcontrollers each have individual power sources. In order to complete the circuit, all grounds have to be connected to each part of the system, adding another layer of complexity to the modularity goal.

Luckily, nothing should explode, and coming soon, a post will be about testing a more modular setup for the LEDs with the entire system, as well as the break down of the code being used.

Testing the Neopixels

During our learning process, specifically investigating the hardware setup, we briefly turned on the LEDs to see the current conditions they were in. Since some of the wires were no longer connected and seemingly in chaotic order, we merely wanted to see if the LEDs even turned on. Indeed the LEDs turned on but were not lighting how we expected: the colors were random and uneven in intensity. The code suggested they all be uniform at least in light intensity if not also in color. So now we came across an issue of whether or not the LEDs were burnt out from the disconnected wiring, miswiring caused by our investigations, power overload, or just bad LEDs.

To find out the cause, we tested each strip individually, with a different arduino from the project, with RGB color strand test code. Through the testing, only one strip seemed to have discoloration and the cause seemed highly probable that the wiring was weak.

This was great news! Wiring is a simple fix and we had already decided to rewire the hardware setup due to inconsistent coloring choices. We wanted to create a more organized system and by doing so, we can confirm if the one LED strip was discolored due to poor wiring or if it is something else.

During this process of testing we were also able to eliminate a possible cause of a low battery supply/overpowered supply. By using a brand new 9V battery, we compared one strip’s intensity between the two batteries. The wires had to be double checked for connections in the process of changing the batteries out. Once all wires were checked to be in order, the LEDs showed no difference in intensity.

Understanding how the system works

Currently, there are two main components to the project: the membrane touchpad and corresponding LEDs. When someone touches a square on the touchpad, the corresponding LED should come on. Eventually, the system will entail changing the LEDs to specific colors, but we are aiming for “on” and “off” phases first.

How this works:

The Membrane Touchpad

The touchpad consists of one “master” Mega Arduino board linked to two “minion” Mega Arduino boards. The master board is set in the middle with one minion board on either side.

Each minion is connected to 30 rows of copper tape, but only at half-lengths; each minion is also connected to 15 full-length columns of copper tape. Minion 1 is connected to columns 1 thru 15; Minion 2 is connected to columns 16 thru 30. “Buttons” are created by the intersections from the copper tape columns with the rows. Each minion is in charge of detecting 450 “buttons.” The minions then calculate the position of the “button” pressed and send this information to the master. This interaction is generated through the use of the keypad library that is available through open-source.

Issue: The keypad library is not available for such a large scale matrix. Our team will have to modify the original library to create an applicable and successful library for our project.

Pictured is a physical picture of the layout of the Master and Minion boards.

Top: Close up of the copper tape connection to Minion 1. Bottom: Close up of Minion 2.

Each minion is connected via I2C bus to the master; the connection is made via pins 20 and 21 on the master: one pin is a data connection and other is clock connection. From our understanding, the I2C bus should prompt an alternating probe between the two minions to the master; the I2C bus is constantly looking for new information while each minion is able to simultaneously report to the master. However, currently the connection is not directly made; the connection is actually made through a breadboard centered in the layout.

Issue: Currently, the I2C bus breadboard setup is not properly connecting both minions simultaneously to the master. As such, the team will have to either solve this issue via software, or develop a new physical setup to properly connect information.

Pictured is the actual connection so far. It is incomplete according to the code.

The LEDs

We have 30 strips with 30 LEDs each (900 pixels total). There are four wires attached to each LED strip: (from left to right) two are ground, one is data, and the last is power.

Two ground wires are required in order to ground the LEDs by two methods: through a breadboard as well as through a power block. The data is currently also going through a breadboard in order to be connected to the master board. The power is connected to the same power block as the one ground wire.

Pictured above is the LED connections to the breadboard. As mentioned already, for each LED, there are two wires going into the breadboard: one ground and one digital.

There are two sizes of power blocks, small (holds two strips) and large (holds six strips), and six power blocks total; from left to right the block are large, small, large, large, small, large. These sizes are chosen somewhat arbitrarily, but the blocks better organize the LEDs while keeping the length of wires in mind. We also have one “mother” power block to channel all of the LED’s power blocks to a power source.

Issue: There are currently not enough power blocks to cover all the LEDs. We will need to gain one more power block in order to cover the last LEDs.

Pictured are the power blocks (Top) and the “mother” power block with its power source connector (Bottom). Currently, the wires connected to the power blocks are all red, which is normally representative of power; but in this case, this is arbitrarily colored and is wiring both power and ground to their respective connections. As an additional note, the capacitors on the power blocks prevent the LEDs from shorting. For now, the amount and strength of the capacitors is a generous guess, but the accurate amount will soon be calculated.

Moving Forward

The team has a few ideas to try out for this semester. We will be trying to match wiring of each power block to individual breadboards and then eventually use permaboard. The permaboard will allow a more permanent system that eventually can become transportable.

Another step towards creating a stable and/or possibly even modifiable system is fixing the the components in place with hot glue or velcro. We can achieve this by laser cutting base pieces to be fastened to the components. In order to better the organization of the system, we will also be breaking up the breadboards, matching the number of the power blocks. As for further developing the LEDs, we need another Mega Arduino or possibly even a Raspberry Pi.

Finally, the team is gaining another member, a computer science major, in order to make progress on the code and connect the LEDs to the entire system!

Membrane keyboard diagram.
Credit: FourOhFour, WikiMedia Commons

Creating this ‘holes’ layer was a seemingly straightforward yet laborious process – cutting 900 holes precisely ain’t easy!

In order to ensure that 30 columns and 30 rows are perfectly exact we used the neoprene cutout stencil to trace and cut squares using an Exacto knife. We also experimented with using a Dremmel cutting attachment, but decided not to use it due to a higher percentage of forced error and lack of safety equipment.

The process loop was as follows:

1. Align stencil with as many cut squares as possible, leaving the left or rightmost columns on uncut membrane space.
2. Trace squares lightly through stencil. Remove stencil and very slowly cut with Exacto knife. If having difficulty cutting, replace with new blade.
   

   Traced squares among cut squares, foam crumbles.

3. Check alignment and proceed to next row or column.

After many hours of work between Mani and me, we completed all 900 holes! Here is the entire grid, some trimming still required:

The almost-complete grid: all 900 holes with some trimming required.

The process of applying the copper tape involved the use of very careful measurements with the aid of the laser cut wooden stencil.

By first layering all three neoprene sheets on top of one another, the corners of the matrix were marked.

Then, using the stencil and a straight edge, the outlines of each individual copper tape strip was sharpied onto each of the two outer neoprene layers.

Once outlined, the copper tape strips were cut to length and then applied to the neoprene. The technique used to apply the copper tapes was similar to that of how a screen protector is applied — peeling the tape away revealing the adhesive side of the copper tape bits at a time as it is applied to the neoprene.

Upon completion of the application process, the final step of the process was to use the middle layer of neoprene (the one that will eventually have 900 1 inch x 1 inch square cutouts in it) to ensure that the copper tapes lined up with little to no offset error.

As outlined in ‘Face’ of the Quilt, I am testing fabric to see how it diffuses, affects affect, and obscures or reveals hardware. I quickly tested a broad array of swatches. Their textures included felt, mid-weight cotton, a polyester-stretch mix, and lightweight linen. The colors included black, brown, grey, beige, and white. The colors are kept neutral because the background of the quilt will be similar to the original beige.

I was first intrigued by how introductory physics principles became useful: light colored or white fabrics tended to diffuse light while dark colored or black fabrics tended to absorb light and reveal the shape of the LED beneath. I demonstrate this as follows:

This gradient of felt (control for weight and texture) shows how lighter fabrics diffuse light while dark colors absorb them

In regards to the LED hue, there seemed to be less effect on the diffusion because there is a high intensity for all colors. Seen here, we see a slightly different diffusion for blue-toned colors but the diffusion remains mostly consistent:

The LED hue has less effect on diffusion due to high intensity (seen here under brown felt)

If I were to pick a personal favorite, it would be the mid-weight cotton-linen mix. The flecks and uneven texture lent it a natural, handmade feel that could be reminiscent of the 1800s quilt – while the modern cool blue light is the perfect juxtaposed touch:

Cotton-linen mix – a very natural, rough feel – juxtaposed with cool blue modern light

For a full tour of the test, we see below the collective effects of different fabric, where color had a greater effect on diffusion than texture:

Fabric Printing

We are investigating printing the Peabody design from an online fabric design service, Spoonflower, instead of sewing the design ourself. The former not only requires less time and money, but may also provide better presentation.

The idea is to design the topmost layer of the quilt in Illustrator exactly to scale and order the design to scale from Spoonflower. The options for the design are as follows:

1. Orange fill on laser-cut design. This layout borrows dimensions from the stencil used to cut holes for the LEDs, such that the grid aligns exactly with where the LEDs shine through. 
2.  Orange grid with major axis, and minor gridlines. This design is closer to the original design that Peabody envisioned.
3.  Orange grid, thin gridlines, less major axis. This is still more similar to the original Peabody design than #1, but reduces emphasis on a divide. Due to the spacing of our LEDs, this is more likely than #2.

Though the online interface for the Peabody quilt allowed for flexibility on the grid design, we are physically constrained by the spacing of the LEDs which do not account for a major axis.

Next, I discovered that the maximum size for Spoonflower designs is about 21″, though you order fabrics in printable areas ranging from 41″ to about 52″. So, we cannot upload our design as-is. Instead, we need to upload 1 corner of the grid and repeat it in a “mirror” fashion:

Spoonflower design interface. Single corner design selected (first row on left). Shown is a mirror pattern that can be used to print a full grid.

Fabric Pricing

I performed a Python web scrape and export to CSV using Spoonflower’s product page to create a straightforward list of every type of fabric offered and the exact cost for our quilt (1 meter). Though, accounting for error and extra fabric, these costs may be higher.

Elizabeth Peabody’s original design

Aspects to consider include:

1. How well do the NeoPixels perform in varying light conditions? Many example projects photograph NeoPixels in the dark. Most likely, we want users to interact with the quilt in partially dim lighting. If necessary, office lighting
2. How will we diffuse the light to create defined borders and retain clarity? In Peabody’s work, richly colored square cutouts were placed on top of a grid. Now, we seek to recreate the effect using LED lights
3. What kind of texture do we want, visually and physically? Should the two mismatch? As a result, how can it create convey a message? (i.e. a soft and smooth texture may convey innocence and warmth and a rough texture may convey stability, trustworthiness, weight)
4. Do we want the hardware to be felt or obscured? Pragmatism may dictate this result, but the difference may determine whether the user perceives an electronic quilt or a different device

In regards to 1. Light conditions and 2. Hardware… We can examine projects made with NeoPixels from Adafruit. The company that created our LED strips publishes many popular example projects that illuminate (heh) the power of NeoPixels:

NeoPixel (our LEDs) glow fur scarf

NeoPixel (our LEDs) bandolier costume

In regards to 2. Defined borders and clarity… we can take a cue from recessed LED light fixtures. Some office buildings opt for LED versus fluorescent lighting fixtures. In these cases, they may use panels of LEDs with polystyrene (PS) diffusing plates of varying thickness. A possible implementation could include a single 1 meter by 1 meter sheet of PS plastic with divisions cut between each touch location to allow for pressing. Examples of LED light fixtures include:

Recessed LED light fixture with polystyrene (PS) square diffuser plate

LED panel diffuser with PS plate

In regards to 3. texture… we have not decided between rough and smooth, but a possible implementation may include rough or smooth cotton fabric. Cotton is affordable, sturdy, and offers a wide range of textures. If choosing a rough fabric, it may be more difficult to find a color that is not a shade of light brown, light green, or in natural shades and block more light. Smooth fabrics tend to come in more modern colors and diffuse light more softly. Examples of fabric include:

An example of a rough fabric; may allow less light to show through

An example of a smooth cotton fabric; may allow more light to show through

In regards to 4. hardware salience… in meetings we have discussed how we might create the “LED sandwich.” A possible implementation is to create “troughs” for the LEDs by placing strips of foam in between each strip of LEDs. This way, the surface will feel roughly uniform, and a potential user may believe a single LED panel lies beneath. However, we may encounter issues with this approach because our neoprene is 1 millimeter thinner than our LED strips, as pictured:

Diagram depicting LED strips with our neoprene foam between; the strips are slightly taller than the foam we purchased

The Peabody Project’s present priority is to complete the physical product. We want to allow for interaction, say in a gallery or exhibition, and sooner than later allow anyone to learn and appreciate Elizabeth Peabody’s contributions to education and data visualization. Combinatorially we already see numerous implementations for our design, each with its own implications for the end product (financial cost, affective experience, etc.). After taking these ideas and experimenting on a small scale, we will be able to determine which path we will take for each.

Thus, we decided to use Adobe Illustrator to generate a stencil that was correctly spaced out. By using the data sheet provided by the manufacturer of the LED strips, we used the values they specified to generate the stencil.

Image of Stencil v1.0 in Adobe Illustrator

After having designed the stencil in Illustrator, we used a laser cutter to cut out the stencil out of plywood. Plywood was used because it was cheap and durable enough to keep its shape when we cut out the neoprene squares.

Our first prototype of the stencil turned out to be off by 0.333 cm between each square and led to the accumulation of an offset error between the LEDs. This error could not have been picked up by simply drawing on the neoprene — which could have led to wasted time and materials. Because we could easily fix this offset error in Illustrator, we had another stencil printed out within a couple of hours and didn’t lose any neoprene in the process.

Neoprene Stencil v1.0

Our final neoprene cutout stencil with correct dimensions is shown below:

Neoprene Stencil v2.0

For our final cutout stencil, we also used a thicker sheet of plywood to ensure the rigidity of our stencil as we cut the squares out of the neoprene.

The power for the strip can come from any end. The +5V needs to be connected to 5 volts power supply. The Ground needs to be connected to both, the ground from the power supply and to the ground pin in the Arduino mega. The data input needs to be connected to Arduino mega through 470 Ohms resistor.

Also the neopixel guide suggests to put 1000 microfarad capacitor in parallel with power. In this case two 4700 microfarads are being used for 10 led strips.

The resistor has been soldered to the wire and is covered by shrink tubing.

In order to power all 30 strips, terminal strips are used to distribute the power. The first terminal strip is connected to the power supply and is used to split the power to another three terminal strips.

Each of the second terminal strips can host 11 connections. 10 are used by the LED strips and then the extra one can be used to power Arduino mega. On the terminals where the power is connected to the strip, two 4700 microfarad capacitors are placed.

The strip itself is separated into positive and negative parts.

The neopixel guide recommends to use 20 – 60 mAmps per pixel. Since there are 900 pixels the total current should be 18 – 54 Amps. But I found that using one 10 Amps power supply is more than sufficient to power all the pixels at about 1/3 brightness. But incase that is not enough a second power supply can be connected to the main terminal strip.