[WIP] Add Pi_grammer programmer

[per1234]

This adds a Tools > Programmer > Pi_grammer menu selection after installing SparkFun AVR Boards, which allows the SPI pins of a Raspberry Pi to be used as an ISP programmer. More info here.

Installation instructions:

1. Install the hourly build of the Arduino IDE. This is only a temporary requirement. Once Boards Manager support for this version of SparkFun AVR Boards is added, it will be usable with any modern version of the Arduino IDE.
2. Download the file avrdude.zip (or if you prefer, build the modified AVRDUDE from source: https://github.com/per1234/avrdude/tree/linuxspi-retry)
3. Unzip the downloaded file.
4. Replace {Arduino IDE installation folder}/hardware/avr/bin/avrdude with the one from the unzipped file (required if you want to use Pi_grammer when the Arduino IDE is not run as root).
5. Download https://github.com/per1234/Arduino_Boards/archive/add-pi_grammer.zip (or clone https://github.com/per1234/Arduino_Boards.git and checkout the add-pi_grammer branch).
6. Unzip the downloaded file.
7. Copy the sparkfun subfolder to {sketchbook folder}/hardware. You can find the location of {sketchbook folder} in the Arduino IDE at File > Preferences > Sketchbook location.
8. Restart the Arduino IDE if it's running.
9. If you have previously installed SparkFun AVR Boards via Boards Manager then you need to remove it so the modified version will be used: Tools > Board > Boards Manager, click on "SparkFun AVR Boards", click "Remove".

Usage

1. Connect your Raspberry Pi to the target board:

RPi GPIO	Target
MOSI	MOSI
MISO	MISO
SCK	SCK
3V3	Vcc
GND	GND
26	RST

Powering a 16 MHz board at 3.3 V is out of spec and may lead to unstable operation. Connecting the RPi GPIO pins directly to a board running at 5 V can damage the RPi so you should use level shifters.

2. Enable SPI on the Raspberry Pi by following these instructions:
   https://learn.sparkfun.com/tutorials/raspberry-pi-spi-and-i2c-tutorial#spi-on-pi
   Note that I found the menu path to be a little different using the latest version of Raspian.
3. (In the Arduino IDE) Tools > Board > select one of the Sparkfun AVR Boards.
4. Tools > Programmer > Pi_grammer.
5. Open the sketch you want to upload to the target.
6. Sketch > Upload Using Programmer.

I've tested this working (every other time, see (1) below) with the SparkFun Digital Sandbox. I haven't tested the 16 MHz boards because I don't have level shifters to safely use my Raspberry Pi with boards running at 5 V.

To do:

• 1. Every other upload fails: avrdude: linuxspi_gpio_op_wr(): Unable to open file /sys/class/gpio/gpio26/direction. (fixed by https://github.com/per1234/avrdude/commit/1b673007e0382564f50aedad8875036ca48de27c)
• 2. The SparkFun pi_program.sh script used for programming their products does a reset of the target between setting fuses and flashing (https://github.com/sparkfun/SparkFun_Pi_AVR_Programmer_HAT/blob/master/pi_program.sh#L29-L33). Due to (1) I have not been able to complete a Burn Bootloader to see if this is necessary. The alternative is to simply set fuses and burn bootloader in a single avrdude command. The two avrdude commands back to back have historically caused problems with the AVRISP mkII (https://github.com/arduino/Arduino/issues/2986) and reportedly continue to do so for the AVR Dragon (https://github.com/arduino/ArduinoCore-avr/issues/9). (Doesn't appear to be necessary)
• 3. The change to platform.txt switches from using using the avrdude.conf bundled with Arduino's AVRDUDE tool to using the avrdude.conf bundled with SparkFun AVR Boards. As reported here: https://github.com/sparkfun/Arduino_Boards/issues/21, this is an outdated version of that file and so this avrdude.conf will need to be updated in order to ensure support for all newer programmers and parts. The current version of the file should be fine for any testing of the Pi_grammer programmer.
• 4. Determine whether there is official interest in adding this programmer to Arduino AVR Boards. Since the proposal comes from SparkFun, I've added the programmer to SparkFun AVR Boards but it could just as easily be done to Arduino AVR Boards instead.
• 5. The default reset pin, as set in the kcuzner/avrdude patch is GPIO 25. SparkFun uses 26 instead and this is the only reason the custom avrdude.conf file is necessary. With GPIO 25 as reset it's possible to use the avrdude.conf bundled with Arduino's avrdude tool and the change to platform.txt is not necessary. It seems like the change to 26 was solely due to the layout of SparkFun's custom hat, which is not likely to be used with Arduino or by general users anyway.
• 6. If this doesn't end up being accepted to Arduino AVR Boards, I think it would actually be more appropriate as a 3rd party package rather than part of SparkFun AVR Boards. This would need to wait for the retry fix to be made to avrdude, the fix to be picked up by Arduino, and Arduino to make a release of Arduino AVR Boards with the fixed AVRDUDE.

Reference:
https://github.com/arduino/Arduino/issues/7795

[lewispg228]

Thanks @per1234 ! Really appreciate your fast response!

I've been giving this a shot, and get the following error:

I think it may have something to do with the version of avrdude that my raspi is using (or rather, which version it is using from the IDE, and which version I can call from a command line.

A couple things:

1. I'm on a fresh install of the latest arduino 1.8.5.

2. I installed the SparkFun Arduino boards (using the boards manager). Then I grabbed the platform.txt, programmers.txt, and avrdude.conf from your fork and overwrote the files here:

/home/pi/.arduino15/packages/SparkFun/hardware/avr/1.1.10

Is this the proper way to try this out?

Also, I tried doing the same calls from command line to see if I could get it to work, and it learned some things.

The exact same call to avrdude (like in the IDE verbose output), will emulate the same error. But then I tried calling straight up "avrdude", and it worked...

What seems strange to me is that the avrdude that came with my recent download is older (?) at 6.3

While the avrdude that is linked to my environment variable "avrdude" is 6.3-20171130

Any thoughts as to what is going on here?

THANKS!!!

[lewispg228]

Also, I just stumbled upon your "every other time it works" problem:

avrdude: linuxspi_gpio_op_wr(): Unable to open file /sys/class/gpio/gpio26/direction

Strangely enough, it will work every time from the command line if the command is preseeded by a "sudo", so I'm guessing it's a permissions thing.

I dug into the folder for the gpio26 direction file, and it's opened up to all users, so I'm stumped as to what is causing this.

[lewispg228]

Update: I currently got it working from the IDE (still fails every other time, but heck, at least the other issue is gone).

It was the fact that the current release of the Arduino IDE comes with older avrdude (6.3). I installed the nightly version and it came with 6.3-20171130, and it worked!

Thanks @facchinm for your comment. It lead me in the write direction!

[per1234]

It was the fact that the current release of the Arduino IDE comes with older avrdude (6.3). I installed the nightly version and it came with 6.3-20171130, and it worked!

Sorry for the confusion. I happened to be using the hourly build of the Arduino IDE. The avrdude version can be specified in the JSON file used for Boards Manager installation of SparkFun Boards. So, as long as the user installs via Boards Manager (which is the only method described in the readme and the guide on the SparkFun website, we can be sure they will always be using the version of avrdude that supports the linuxspi programmer type.

I did a bit of research on the avrdude: linuxspi_gpio_op_wr(): Unable to open file /sys/class/gpio/gpio26/direction error. The only relevant result I found is this:
https://www.avrfreaks.net/comment/2466356#comment-2466356
but that's only a report of the problem, no solution.
I'm not very experienced with Linux and this is my first time ever working with the RPi's GPIO so I'm out of my depth here. I find that before the error occurs the /sys/class/gpio/gpio26 folder does not exist, then on the run of avrdude that produces the error that folder appears and so the next run of the command is successful. I believe this could easily be worked around by bundling a wrapper script with SparkFun AVR Boards, e.g.:

# set up GPIO 26 for use to reset the target
echo 26 > /sys/class/gpio/export
# run avrdude command...

but I really like the current very minimal implementation. The script workaround would require adding a new tool to platform.txt that would be used only for the Pi_Grammer programmer so it's kind of messy. So I'd prefer to find a solution using the existing avrdude tool definition but I'm also stumped.

[lewispg228]

@per1234 Thanks for looking further into this.

The avrdude version can be specified in the JSON file used for Boards Manager installation of SparkFun Boards

Interesting. Where does/would it install this on the pi? I was searching around the packages folder (/home/pi/.arduino15/packages/SparkFun/hardware/avr/1.1.10) and couldn't see it anywhere. I'm new to all this, and it seems a little tricky to keep track of where all the actual avrdude versions and settings files live (programmers.txt, avrdude.conf, etc.)

I will ask around here at SparkFun to see if anyone knows more about this gpio issue. Really appreciate your help so far!

[lewispg228]

I haven't tested the 16 MHz boards because I don't have level shifters to safely use my Raspberry Pi with boards running at 5 V.

Just an FYI here, I have been testing this out with a little custom programming HAT (which include level shifters), and it is working well on a 5V/16MHz board (The SparkFun Redboard).

[per1234]

Where does/would it install this on the pi?

It would be installed to /home/pi/.arduino15/packages/tools/avrdude/6.3.0-arduino14

I will ask around here at SparkFun to see if anyone knows more about this gpio issue.

Great! Hopefully someone with some more RPi experience than I can point us in the right direction.

I have been testing this out with a little custom programming HAT (which include level shifters)

I really enjoyed reading your article about the RPi standalone programmer.

[per1234]

I found the solution to the avrdude: linuxspi_gpio_op_wr(): Unable to open file /sys/class/gpio/gpio26/direction bug. When the GPIO file is created it initially belongs to root but there is a udev rule that changes its ownership to the gpio user group. The problem is it takes a little bit of time for that to happen but avrdude tries to set the GPIO 26 direction immediately after exporting it. At that time it is still owned by root and so this is why the process was failing when not run as root. The solution is that if the file open process fails avrdude needs to wait a bit, then try again.

It turned out someone had submitted a patch to the linuxspi author two years ago for this:
kcuzner/avrdude#10
but unfortunately it has not received any attention.

I improved the patch a bit, applied it to the latest Arduino version of avrdude:
https://github.com/per1234/avrdude/commit/bd1d6dd432f8a910e67b2ec8561b1f72a654925f
and compiled it. Now the Pi_grammer programmer selection works great for me in the Arduino IDE even when I'm not running it as root.

You can download the modified avrdude here: https://github.com/per1234/Arduino_Boards/files/2226696/avrdude.zip or build it from my source if you prefer: https://github.com/per1234/avrdude/tree/linuxspi-retry (here's the build script that applies Arduino's patches: https://github.com/arduino/avrdude-build-script/tree/staging). Then just replace {Arduino IDE installation folder}/hardware/avr/bin/avrdude with the modified version.

I find that Burn Bootloader works fine now so I don't think any modifications need to be made to deal with the current back to back avrdude commands. It would be great if others would give it a try and let me know how it works for them.

So now the question of how to proceed? I'm going to submit a PR (kcuzner/avrdude#17) to kcuzner/avrdude and would appreciate any feedback on that code as I have no prior experience with this. After that is requesting Arduino to update their AVRDUDE code according to the updated patch. If SparkFun wants to proceed with this project, it's not actually necessary to wait on any of that since they have the option of simply distributing their own version of the AVRDUDE tool via the Boards Manager installation.

[wraybowling]

I'm trying to follow the breadcrumb trail to get my pi zero working as a programmer today. It appears that kcuzner/avrdude#17 was merged, so if there's no more action to be taken, I'd like to see this PR closed so no future person tries to follow it.

[per1234]

What is the harm in someone following it?

[wraybowling]

🤷🏻‍♂️ if it's not abandoned work, nothing's wrong. Looked like you solved the issue elsewhere though.

[per1234]

if it's not abandoned work, nothing's wrong

It's not abandoned. I would like to see this programmer made easily accessible for people to use via the Arduino IDE. It is true that I don't know when I'll have time to move forward with this.

I had planned to make a 3rd party platform that would allow people to install support for the programmer via Boards Manager. I had to wait for my patch to AVRDUDE to be merged, and for Arduino to release a version of AVRDUDE that contained that patch. That took a long time. By the time it did happen, health issues and the associated ongoing medical expenses have made it so that I now have very little time for volunteer work.

Now, Arduino has made a change to the Arduino IDE that means 3rd party programmer platforms will no longer be possible: arduino/Arduino#9900

So the only path forward is to submit a pull request to Arduino AVR Boards and any major 3rd party AVR-based platforms that don't reference the arduino core. This PR shows what would be needed to do that, so it's very much still relevant.

The intent of this PR was to demonstrate the necessary changes and provide a place for me to discuss them with lewispg228. For the actual PR, I would change the programmer name to something standardized like linuxspi. I would also forget about using a custom reset pin. It's not like there are a lot of people using the Pi hat SparkFun made for their internal usage which requires this change, and a custom reset pin will cause a lot of inconvenience, since some platforms don't bundle a custom avrdude.conf. Better for everyone to standardize on the default pin 25.

Looked like you solved the issue elsewhere though.

The sub-issue of the bug in AVRDUDE has been solved, at least in Arduino's fork. But the AVRDUDE issue was only a blocker for the real goal of this project: to make the linuxspi programmer available for use via the Arduino IDE. That is very much not solved yet.

[per1234]

I'll share an update: the linuxspi programmer was finally added to the official AVRDUDE in avrdudes/avrdude@40b0b10 and released in 6.4

The implementation is a little bit different from the original.

The good news is that it is not affected by kcuzner/avrdude#10.

The bad news is that the port name format is different. So the -P value must be changed from /dev/spidev0.0 to /dev/spidev0.0:/dev/gpiochip0.

The -b flag has been deprecated in favor of a -B flag: avrdudes/avrdude@2aee540. There is a backward compatibility conversion in place which is intended to make the old style -b 2000000 equivalent to -B 2MHz. The conversion is currently not working (avrdudes/avrdude#927), but there is already a proposed fix.

Other than that, I have found the official linuxspi to work perfectly.