New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alternative bootloader? #25

Open
perillamint opened this Issue Jul 14, 2017 · 84 comments

Comments

6 participants
@perillamint
Copy link

perillamint commented Jul 14, 2017

Quite out of scope of this project, but as all we know, default DFU bootloader is quite unstable. It does not work "well" on Mac or Linux.

I think alternative bootloader which supports standard DFU protocol can help firmware development in Linux or other non-Windows systems.

Could we build alternative bootloader which supports DFU protocol other then default buggy USB-MSD DFU?

@Ralim

This comment has been minimized.

Copy link
Owner

Ralim commented Jul 14, 2017

Hi,
I'm completely open to building an alternate bootloader for the unit. I toyed with this before, however, most people seemed to prefer the MSD based bootloader for ease of use. Also considering the number of emails, messages and issues that are similar to #23 I don't overly want to advertise it too heavily.

It would certainly be possible to implement an alternate bootloader & then have a conversion firmware that swapped out the bootloaders. If you would find this useful though, I can start a GitHub project for this and try and get a replacement bootloader up and going.

@perillamint

This comment has been minimized.

Copy link

perillamint commented Jul 18, 2017

I think I could get some time to tinker with iron firmware 1-3 months later. (need to source another iron for SWD pin soldering, get enough time to spend hacking new bootloader, etc)

However I have one question before wiping out bootloader of my iron (or another TS-100): Could I get back into stock system when I flash "DO NOT FLASH_Flash Backup.hex" using ST-Link?

@Ralim

This comment has been minimized.

Copy link
Owner

Ralim commented Jul 18, 2017

That would be awesome!

That is exactly the purpose of that file, it's an entire read out of the entire flash of my iron when I bought it (original fw 2.12). For if someone did manage to wipe theirs :)

@Ralim

This comment has been minimized.

Copy link
Owner

Ralim commented Aug 4, 2017

So I'm thinking of forking dapboot and working to get that running on the iron. So that we can use web-dfu along with the normal dfu-util to update the iron.

Mostly going to need a fair bit of testing before I push anything too publicly.... Dont want to brick irons :)

@vifino

This comment has been minimized.

Copy link

vifino commented Aug 4, 2017

I am also in favor of "normal" DFU, to be used with dfu-util. Much easier, IMO.
Once mine arrives, I'll be happy to test. Got JTAG/SWD adapters and everything, so in case something goes seriously wrong, I'll be able to recover.

@Ralim

This comment has been minimized.

Copy link
Owner

Ralim commented Aug 4, 2017

the dapboot implementation supports both web based and the normal dfu-util tool :)
Awesome, will be good to have another tester :)

@Maartenwut

This comment has been minimized.

Copy link

Maartenwut commented Aug 5, 2017

I'll be glad to have another bootloader as well - I'm in for testing!
There's just one thing: How are we going to get to the pins? They are too small to solder without the risk of ruining everything.

@perillamint

This comment has been minimized.

Copy link

perillamint commented Aug 12, 2017

I purchased Olimex STM32-H103 board and flashed that "DO NOT FLASH_Flash Backup.hex". After some configuration (Pull down PC11), It boots successfully on that board.

I think I could start bootloader development on this board rather then directly on iron.

@Ralim

This comment has been minimized.

Copy link
Owner

Ralim commented Aug 12, 2017

Hey,
Yeah I have been using one of the blue common f103 breakout boards to work on. I just haven't had much time. The only issue I could see with this is getting the bootloader working well without the crystal. As I'm not sure if it just works at room temperature, or if they are performing a calibration on the internal oscillator. Since I think the internal oscillator tolerance is wider than USB technically requires.

I am planning to fork dapboot for the iron as it supports the web dfu, and see if it can run without the crystal. (The ts100 does not have any crystals or timing references fitted)

@perillamint

This comment has been minimized.

Copy link

perillamint commented Aug 12, 2017

@Ralim I tested it using rcc_clock_setup_in_hsi_out_48mhz() instead of rcc_clock_setup_in_hse_8mhz_out_72mhz(). It enumerates well and works well.

Host used: Dell XPS 9550

Here is my fork of DAPBoot https://github.com/perillamint/dapboot

@Ralim

This comment has been minimized.

Copy link
Owner

Ralim commented Aug 12, 2017

Awesome, if you want to develop it in a branch of this repo I can give you push access if that's easier? (Lets us keep the code for the iron fairly central). Thoughts ?

@perillamint

This comment has been minimized.

Copy link

perillamint commented Aug 12, 2017

BTW, I tried to load original firmware using DAPBoot. It seems loads find but Stock firmware's USB does not enumerate with error -71

I suspect clock configuration but I'm not sure. Does anyone know any clue about this?

@perillamint

This comment has been minimized.

Copy link

perillamint commented Aug 12, 2017

Hey @Ralim, Could you make documentation about pads on small STM32F103 board in TS100?

I'm planning to get another TS100 in September and that information will be great help

@Maartenwut

This comment has been minimized.

Copy link

Maartenwut commented Aug 12, 2017

untitled-1

@vifino

This comment has been minimized.

Copy link

vifino commented Aug 12, 2017

Wow!
Looks like some progress happened!

My iron will be here in the next couple of days - tell me if you need testers.

@Ralim

This comment has been minimized.

Copy link
Owner

Ralim commented Aug 13, 2017

Hi all,
@perillamint I'll try and document the pads for you. I recently picked up another one from work for working on the bootloader. I'm 99% sure it will be the clock configuration. The firmware expects the bootloader to setup the clocks for 72MHz. Also ensure the bootloader is compiling and fitting, as the firmware expects to start at 0x8004000 off memory (so therefore size of the bootloader should be < 0x4000).

The 4 pads on the bottom of the board are GND,VCC,SWDIO,SWCLK. In that picture off memory they are GND in bottom left, vcc in top left, and the other two are clk/dio (i just swapped jumpers till it worked).

Once we get the bootloader up and running we can make a conversion firmware that re-flashes it using the old bootloader.

@perillamint

This comment has been minimized.

Copy link

perillamint commented Aug 13, 2017

I dumped rcc memory regions and it differ quite a bit. I'll try document this changes and apply this hack on bootloader.

I think this change could not merged back into dapboot mainline, so I think only option is fork and maintain dapboot for TS100.

@Ralim

This comment has been minimized.

Copy link
Owner

Ralim commented Aug 13, 2017

Yeah, I agree that we will have to maintain our own fork of dapboot.
Should not be too hard to maintain, and will just need to keep an eye on what happens to it.

@perillamint

This comment has been minimized.

Copy link

perillamint commented Aug 16, 2017

My backup iron arrived today. I soldered ST-Link and flashed my bootloader.

It successfully loads stock firmware (without USB) and OLED shows text on it. However when I put your firmware, it does not turn on display.

Do you have any idea about it?

@Ralim

This comment has been minimized.

Copy link
Owner

Ralim commented Aug 16, 2017

Interesting,
I'm using the same initialisation code as they do for the clocks etc, does your code jump to 0x4000?

Not massively sure what would cause it to hang, I'm happy to try on my development iron shortly (hectic week at the moment).

Can I ask which firmware you loaded? Also if you can inspect the program counter on the crash. My current best guess would be that the bootloader is not placing the firmware at 0x4000 offset so when my code sets the nvic to that offset the interrupts get invalid addresses for the Irq handlers.

Off a very vague memory the bootloader uses 0x8000 as it's loading point instead. I probably could technically drop the nvic remap in my code as I'm fairly sure the official firmware does it as well. This would also make sense as I make heavy use of the interrupts where as their code was entirely software polling based except for the usb connection off memory.

@perillamint

This comment has been minimized.

Copy link

perillamint commented Aug 16, 2017

I succeeded! The problem was this: eDesignOSS/dapboot@9966fbd

TS100 stock firmware assumes its bootloader turned on some STM32 peripherals. After I turn on USB and I2C, it works just fine.

@perillamint

This comment has been minimized.

Copy link

perillamint commented Aug 16, 2017

Now, we need some way to flash patched DAPBoot bootloader using stock bootloader.

@perillamint

This comment has been minimized.

Copy link

perillamint commented Aug 16, 2017

BTW, DAPBoot is not quite "fool proof" -- it could brick itself when user put large blob on it. I think we should mitigate this problem.

@perillamint

This comment has been minimized.

Copy link

perillamint commented Aug 17, 2017

I fixed self-brick issue with eDesignOSS/dapboot@e3a5a63 this commit.

IMHO, After developing user-friendly way to replace bootloader using stock bootloader(maybe some kind of bootloader updater firmware?), we could do some beta(or alpha?)-testing from that point.

What do you think @Ralim?

@Ralim

This comment has been minimized.

Copy link
Owner

Ralim commented Aug 17, 2017

Hi,
Can you confirm if this bootloader protects you from trying to flash to anywhere below 0x8004000?
I'm happy to make up some conversion firmware that would re-flash the bootloader. Just rather busy at the moment.

@perillamint

This comment has been minimized.

Copy link

perillamint commented Aug 18, 2017

@Ralim Since DAPBoot does not support DfuSe, I don't think user could put negative number on current_dfu_offset

https://github.com/devanlai/dapboot/blob/master/src/dfu.c#L232

However, if we need real fool-proof, I think I could extend this patch(eDesignOSS/dapboot@e3a5a63) little bit to implement only allows flashing command within this address range: (0x08000000 + APP_BASE) - (0x08000000 + FLASH_SIZE)

@perillamint

This comment has been minimized.

Copy link

perillamint commented Aug 18, 2017

BTW, I think I could work on bootloader flasher in next week (or 2-weeks later, depends on my other works :)).

Since stock bootloader is quite fragile, I think bootloader flasher should implement some safety mechanism like payload checksum check to protect devices from "hard brick" scenario.

Another question is: Should we "merge" DAPBoot fork into your firmware repository? or should we create GH group to maintain TS100 related repos? What do you think @Ralim?

@Ralim

This comment has been minimized.

Copy link
Owner

Ralim commented Aug 18, 2017

Good to hear, I wasn't sure of memory how much protection it had for trying to flash the wrong thing :) Mostly just asking :)

It might be worth adding the extra protection just in case, can't hurt to add it :)

Yeah, I'm hoping I'll have more time soon to keep working on the firmware for this, just been a bit too busy with working. The .HEX file itself has a rough checksum in it, and the dfuse protocol has checksums from memory. The biggest issue is not corruption in the file transfer but just getting the file there, which the dfuse should solve.

We cant really stop people flashing the wrong files, we just have to make it so that those diles cannot break the dfu bootloader :)

I'm thinking of making a Github org for community E-design products since I'm slowly working on a codebase for the ES120 as well. Then we could use that to organise all the future repositories? (Might not move this one for a bit since this link has been shared a fair bit). Thoughts?

@perillamint

This comment has been minimized.

Copy link

perillamint commented Aug 18, 2017

I think cloning this repo to new group and put migration notice on this repo will be enough for redirecting people to new repo.

BTW, I'm planning to buy ES120 next month. It looks quite nice. I guess this DAPBoot port will work on ES120 without huge effort. I hope they didn't lock flash on ES120 like TS100 did. :)

@perillamint

This comment has been minimized.

Copy link

perillamint commented Aug 18, 2017

Just FYI, I'm trying to mainline some of the patches from DAPBoot fork to original DAPBoot. Although we could not mainline all of our changesets, some parts like flash overrun protection will be helpful to other hardware.

https://github.com/devanlai/dapboot/issues

@vifino

This comment has been minimized.

Copy link

vifino commented Sep 19, 2017

I'll flash it after I cleaned up my mess here. Will be in a couple hours at max.

@vifino

This comment has been minimized.

Copy link

vifino commented Sep 20, 2017

@perillamint Oops, forgot to report back.

It flashed successfully, DAPBoot loads. Now to flash back Ralim's firmware.

Edit: Tried to flash a Hex, while dfu-util is expecting a binary. Oops.

@Ralim

This comment has been minimized.

Copy link
Owner

Ralim commented Sep 20, 2017

@vifino Just for the sake of it, have you tried the web-based flashing tool?

https://devanlai.github.io/webdfu/dfu-util/

@vifino

This comment has been minimized.

Copy link

vifino commented Sep 20, 2017

@Ralim No, but I know what I am doing wrong. I was feeding dfu-util a .hex, not a binary.

@Ralim

This comment has been minimized.

Copy link
Owner

Ralim commented Sep 20, 2017

Ah okay, yeah I don't currently push up .bin files to avoid confusion to users.
There are converters around or I can provide some .bin files, later on, today if you need them.

@vifino

This comment has been minimized.

Copy link

vifino commented Sep 20, 2017

If you could poke me with a .bin, it'd be much appreciated, @Ralim. Currently, I am stuck with a windows machine, so there isn't much I can do.

@Ralim

This comment has been minimized.

Copy link
Owner

Ralim commented Sep 20, 2017

TS100.zip
This is the V2.x release currently, so not perfect and may have some bugs but will let you get back to a working system hopefully and also let you test the bootloader.
Ill get some .bins for the latest 1.x release up tonight

@perillamint

This comment has been minimized.

Copy link

perillamint commented Sep 20, 2017

I just woke up. @vifino Did it work well?

@perillamint

This comment has been minimized.

Copy link

perillamint commented Sep 20, 2017

BTW, we have 3 repos for this project. After we got ES120 screwdriver, It will become 4.

How about migrating this repo and my two repos to GitHub group? @Ralim

@Ralim

This comment has been minimized.

Copy link
Owner

Ralim commented Sep 20, 2017

@perillamint Yes, I will make an organisation shortly, any idea what the best name is?
Currently not sure if eDesignOpenSource or something is worth it or? (Since don't want to use TS100 as it's not the only product really)

@perillamint

This comment has been minimized.

Copy link

perillamint commented Sep 20, 2017

@Ralim Their company name is MiniDSO and they use MIN as their logo, so I suggest OpenMIN as our org name. I don't have any better idea about this. Out of naming creativity :'(

@Ralim

This comment has been minimized.

Copy link
Owner

Ralim commented Sep 20, 2017

E-design is the parent to miniware aka minidso.
They use E-design for the es128, but miniware on the ts100 oddly.

So really not sure where to go :P

@perillamint

This comment has been minimized.

Copy link

perillamint commented Sep 20, 2017

Maybe eDesignOSFW? or eDesignMod?

I got this two in my head now

@Ralim

This comment has been minimized.

Copy link
Owner

Ralim commented Sep 20, 2017

eDesignOSS?

@perillamint

This comment has been minimized.

Copy link

perillamint commented Sep 20, 2017

eDesignOSS. That looks good.

@perillamint

This comment has been minimized.

Copy link

perillamint commented Sep 20, 2017

OK. I transfered my two repo into eDesignOSS org.

@Ralim

This comment has been minimized.

Copy link
Owner

Ralim commented Sep 20, 2017

I'll transfer this one as soon, just need to set aside time to clean up some stuff first :)

@vifino

This comment has been minimized.

Copy link

vifino commented Sep 20, 2017

@Ralim It worked, thank you!

I do have a few issues with it, however. It seems very, very slow. Having to hold the buttons to get them to register (~0.5-1s at least), OLED visibly redrawing in settings. Other than that, it looks quite cool.

@vifino

This comment has been minimized.

Copy link

vifino commented Sep 20, 2017

@perillamint It did! Even my stupidity with trying to flash an intel hex encoded file instead of a binary didn't break it!
Thank you so much for your work! Made things much easier for me.
Note to self: Don't DFU at 4 am.

@perillamint

This comment has been minimized.

Copy link

perillamint commented Sep 20, 2017

@vifino BTW, You could use http://hex2bin.sourceforge.net/ to convert hex to DFU-able bin. I converted recent 1.17.1 HEX into BIN. If you want a stable version, here is the file which I converted using hex2bin. https://www.dropbox.com/s/8p8i21az24q72zo/ts100.bin?dl=0

@vifino

This comment has been minimized.

Copy link

vifino commented Sep 20, 2017

@perillamint Thank you! I tried to do it using srecord, resulted in a huge binary, which obviously didn't work.
I really appreciate you handing me the bin, and of course your work. :)

@Ralim

This comment has been minimized.

Copy link
Owner

Ralim commented Oct 3, 2017

@vifino Yeah sorry that bin file I uploaded was broken :/ oops.
I'm uploading .bin files of the newer firmware as I go as well, so feel free to try them :)

@vifino

This comment has been minimized.

Copy link

vifino commented Oct 3, 2017

@Ralim I'll surely do so. Probably not now though. It's 7 am and I've been up all night learning Lojban. ^_^

@vifino

This comment has been minimized.

Copy link

vifino commented Oct 3, 2017

@Ralim I've just tried it again, muuuuch better!
I like it a lot sofar. I am not sure if I imagine this, but could it heat just a tiny bit faster? It doesn't overshoot but reaches the temperature perfectly quite quickly. (On 24V)
The first half-second it gets confused, however, it displayed temperature way past 500 degrees celsius, but then it comes back to ~40 degrees and everything works fine again. I've ran a tip calibration before that.

I've also noticed that the help string for the calibration is cut off when it loops around. Buffersize too small? You could determine the appropriate buffersize by using macros on all translation help strings, perhaps.

When there is a temperature warning, it seems to not be centered.

This 2.0 is quite an upgrade. One thing I did not find was the power source voltage calibration. Luckily my iron is quite accurate by default. I like the new idle screen. Having the advanced display not be used in the actual soldering mode is a bit unfortunate, because I'd like more information while soldering, not while in idle.

That's all the issues I could find in what limited testing I've done. Keep up the great work!

@Ralim

This comment has been minimized.

Copy link
Owner

Ralim commented Oct 4, 2017

Hi @vifino,
Ah, okay looks like the PID has a little bit of wind up somewhere, I'll look into that.

Also, another thing to note is that there is a rolling average filter on what is displayed on the screen, so when the screen is showing that it's at temp it has probably been at temp for around 1 second at the temperature sensor. This is done because the temp sensor is a little bit away from the tip itself, so the idea is that by the time that filter is showing it at temp, the tip should have equalized to that same temperature. The filter is set to around 0.8 seconds as this seems to be about the equalization time at a guess from asking some other people 😄

It shouldn't be showing way past 500 C normally unless the tip is unplugged. I'll try and re-create this locally as well.

Oops, thats probably a lack of padding in the wraparound :/. All the strings are using macros for string lengths etc. I use strlen atm

The centering of the temperature warning page is dependant on the length of the text used for the warning (language dependent). As currently all text is drawn left to right.

The 2.x firmware is still in alpha and voltage calibration is still on the todo list 😄

What would you like to see in the advanced soldering screen? More than happy to add one, just wasn't sure what should be there?

@thanks4opensource

This comment has been minimized.

Copy link
Contributor

thanks4opensource commented Oct 5, 2017

@Ralim: Continuing to move off-subject for this issue, and you weren't asking me, anyway ;) ...

The 2.0 alpha's PID behavior subjectively seems a little laggy compared to 1.7. Nothing I can prove, and maybe I'm imagining it. In terms of display, at least in "ADVDSP F" it seems like the power bar graph (which replaced the squiggly heater lines which always looked like bacon to me) used to briefly spike to full coming out of sleep mode, like I'd expect it to, but doesn't anymore. Don't know if that's the PID changes, display hysteresis, or something else.

As far as tip vs sensor temperature deltas go, I've been thinking that the very end of the tip, at least with the small ones like the "I" and "ILS", is cooler than the more massive body. Makes sense, assuming the heating element is farther away and the small end has more surface area vs volume to radiate from. I have no experience with other high-end soldering irons, but I'm wondering if there's a plating quality issue with the TS100 tips. I'm trying to learn fine-pitch SMD soldering and common advice is to put a tiny blob of solder on the very end of the tip and then touch that to a well-fluxed chip lead and PC board pad.

Problem with at least my "I" and "ILS" tips is when I try to do that the solder "wicks" up to the larger (hotter?) main body of the tip and none of it sticks to the very end. Apologies for this long rant, but bringing it back to firmware and PID algorithms, could there be any way to track the observed sensor time-vs-temp response curve and adaptively tweak the PID constants in real time to try to adjust for different tip's thermal behavior (and environmental conditions)?

@vifino

This comment has been minimized.

Copy link

vifino commented Oct 5, 2017

@Ralim No, I mean, it heats a bit faster, I believe. :)

strlen does not sound like macros... But it doesn't matter, those are micro optimizations for things that don't really matter, the chip in the ts100 is more than capable.

Regarding the temperature display jump in the first few fractions of a second, I believe it could indeed be just a little disconnect for a few milliseconds. I tried to reproduce it again, but it worked fine now.

Interestingly, my issue regarding text overwriting also disappeared... Now it's just starting over like it should.

Not sure if it is my iron acting up, the firmware having bugs in it or a faulty supply/flickers...

Seems kinda unlikely for that stuff to happen, might just be imagining things, so take my input with a grain of salt.

@vifino

This comment has been minimized.

Copy link

vifino commented Oct 5, 2017

Oh, and regarding the advanced display for the soldering mode: Similar to what you have now in the main screen, voltage, temperature, set temperature, heating output in percentage, stuff like that.

I also figured out the weird 500 degree celsius issue by enabling the advanced display: It was a bad calibration. Might have done the initial calibration on USB power only. Could explain some things.
Now it's all good. Just the description text remains a mystery to me.

@Ralim

This comment has been minimized.

Copy link
Owner

Ralim commented Oct 5, 2017

@thanks4opensource

The PID should be faster, but the whole system is different style (it's now uncoupled and different non-syncronized timing). So not really sure how to compare that honestly 😄
The not showing the heat icon is most likely down to a bit of PID tuning (The PID is currently tuned for good stability at the cost of a slight increase in response time). Should still be damn quick from my testing, but if its an issue let me know and I can spend another night tuning 😁

I don't think its a plating quality issue, I use the C1 (ILS's twin) for all of my SMD and I have found the slight slant on the tip helps, but really what we need for soldering SMD is a J tip rather than the fine ones. I would advise looking at the "tack a pin, flood, solder wick" style where possible 😄
I can say that the C1 tip is a lot nicer than most of the Weller tips I have on my WESD51 on the other bench that's for sure. One trick I use is to have the blob on the side of the tip and bring the tip down parallel to the pin from above sometimes works well, oh, and flood with flux 😁

Basically, yes but its hard and I don't have the time to spend on that to gain ~0.5 seconds of faster heating on an iron that already beats a lot of them. The payoff usually seems too small for me to put the time into an autotuner that doesn't get in the way.

Currently, I tune for the larger tips but I check on my C1 tipped iron before I ship it (as I use that iron more, but the development iron has a B2 tip in it).

@vifino
Thanks, that's what I was hoping to achieve in 2.x

strlen is not a macro but its a tiny cost for being a lazy programmer 😛

If you chuck your ideas for an advanced soldering display into its own issue Ill add it in the next 2.x alpha if I can :)

USB power shouldn't affect the calibration, but It does need the tip to be equal to the handle temp, so if you have been holding the handle for ages, for example, its a bit out of whack (though ~5 C is nothing really on the temps you normally solder at, but accuracy is nice to have).

I will be spending more time on the calibration stuff, and so as more users use the feature I will look to see how it goes.

500C is odd, since the way the electronics are set up we can only measure up to ~465C (Thus the 450C hard limit). Normally if the unit sees > 450C for more than ~ 1 second it should trigger disconnected tip so not sure how that happened :s

@vifino

This comment has been minimized.

Copy link

vifino commented Oct 5, 2017

@Ralim I don't know either, but a recalibration fixed it, so that's all I care about. :)

While I would always love faster/better/fancier things and features, v2 definitly satisfies most if not all my needs and wishes. I do look forward to what you can come up with, definitly including the advanced solder mode, as I believe it to be much more useful than the regular one, voltage sag would be nice to notice, or a lipo's voltage. :)
Please do make it seperate from the ADVDISP thing, as I do not really like it most of the time, since it does not give me information I need pre-soldering.

Of course, if you find a way to magically make it even faster/accurate at holding temperature/learning from reaction of the tip, it'd be insanely awesome, but really, it's definitly not required as it is already very good.

I usually use the standard or the ILS tip, as I am doing work ranging from small (building quadcopters and fixing simpler things) to tiny (SMD soldering and such, mostly fixing). Definitly works fine!

Keep up the great work! I might contribute in the future, but so far, I don't see anything I consider to be important to be fixed right away. (That's a very good thing!)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment