TSDZ2 EBike wireless standard (like Specialized Turbo Levo) - OpenSource

[casainho]

-
I think the bootloader is the more urgent task so anyone can already build his remote and easily update future firmware improvements. Right now this remote is already great for Garmin Edge users!

Thanks for the merge!
I will start working on the Bluetooth bootloader now.

Please before start implementing, write here the path you plan to follow. I implemented a bootloader before for NRF52 based on button press at startup and their example code... But our case is a bit more specific and I wonder how do you think it can work on our case.

 

[rananna]

My plan is to implement a 'buttonless' bootloader than can be activated in DFU mode using only a software flag.
That way you do not have to press any buttons on the NRF52840 dongle for updating the motor firmware.

The software flag could be set using your Android app for the motor , or by using a long button press (say 10 seconds) on the remote control.
Once the bootloader is in DFU mode, a packaged firmware file could be used to program a new firmware.
I plan to have version control on the packaged update firmware files as well.

 

[casainho]

My plan is to implement a 'buttonless' bootloader than can be activated in DFU mode using only a software flag.
That way you do not have to press any buttons on the NRF52840 dongle for updating the motor firmware.

The software flag could be set using your Android app for the motor , or by using a long button press (say 10 seconds) on the remote control.
Once the bootloader is in DFU mode, a packaged firmware file could be used to program a new firmware.
I plan to have version control on the packaged update firmware files as well.

I like the idea of the button less. Maybe the bootloader could be activated when the ANT ID change is also activated? this way the enable the Bluetooth decision would be the same, that you already implemented.

I see a big problem with button less, that is it is implemented on the user firmware that means a bug or a version of firmware without this feature and not more access to the bootloader!!

I think that maybe the best way would be button less on the user firmware + regular bootloader that could be activated at startup when user is pressing all the buttons simultaneous, like checking after boot + 500ms for the all buttons active to activate the bootloader otherwise jump to user fimware. But if you see, I think button less is not needed if the button mode works.

 

[rananna]

I like the idea of the button less. Maybe the bootloader could be activated when the ANT ID change is also activated? this way the enable the Bluetooth decision would be the same, that you already implemented.

We could certainly easily reboot into the bootloader on the ANT ID change, but we would not want to enter DFU mode on every boot. We still need a flag to tell the bootloader to enter DFU mode.

I see a big problem with button less, that is it is implemented on the user firmware that means a bug or a version of firmware without this feature and not more access to the bootloader!!

Actually, this will not be the case. I will program the new bootloader features to be a superset of the original button enabled bootloader.
That is, the new bootloader will be able to work either with a button, or buttonless.
That way, if you are stuck and cannot get the bootloader into DFU mode wirelessly, then you can always revert to the older button based method of the original bootloader

I wanted wireless bootloader to simplify the firmware updates for both the motor and the remote control.

 

[rananna]

I like the idea of the button less. Maybe the bootloader could be activated when the ANT ID change is also activated? this way the enable the Bluetooth decision would be the same, that you already implemented.

We could certainly easily reboot into the bootloader on the ANT ID change, but we would not want to enter DFU mode on every boot. We still need a flag to tell the bootloader to enter DFU mode.

thinking about this some more, we could write a special ant ID that sets bootloader mode.
any other number would just set the ANT ID. Only one number would cause a reboot into DFU mode.

So, what do you think?
Should I start implementing this?

 

[casainho]

I like the idea of the button less. Maybe the bootloader could be activated when the ANT ID change is also activated? this way the enable the Bluetooth decision would be the same, that you already implemented.

We could certainly easily reboot into the bootloader on the ANT ID change, but we would not want to enter DFU mode on every boot. We still need a flag to tell the bootloader to enter DFU mode.

thinking about this some more, we could write a special ant ID that sets bootloader mode.
any other number would just set the ANT ID. Only one number would cause a reboot into DFU mode.

So, what do you think?
Should I start implementing this?

On last days I have difficulties to access the forum. The only way it works for me, as right now, is to use TOR browser....

See that there is the examples/ble_peripheral/ble_app_buttonless_dfu bootloader: https://infocenter.nordicsemi.com/index.jsp?topic=%2Fcom.nordic.infocenter.sdk5.v15.2.0%2Fble_sdk_app_buttonless_dfu.html

That one is the most advanced button less, as the bootloader is always active when the user firmware is running, meaning no need to button press or any command.
But this should be complex, I think we do not really need it.

Yes, I also think the bootloader should be common as possible and so I created a new repository for that - please make your pull requests to here: https://github.com/OpenSource-EBike-firmware/TSDZ2_wireless-bootloader

So, about the bootloader you want to implement, it think will work like this:

Enter bootloader method 1: button press at startup

TSDZ2_wireless: at startup, board button long press of 4 seconds (because we have only the same button to startup, a long press need to be detected - after 1 second, if no button press, start user firmware otherwise keep counting the time until reach no more than 4 seconds to enter bootloader -- this time of 1 second will add to delay of system startup...)

wireless_remote: all buttons long press of at least 1 second (1 second long press so user will not hit both at the same time but at 1 second it should already do it)

Enter bootloader method 2: command from user firmware
TSDZ2_wireless: just like ANT ID, user can set a value > 0 to specific Bluetooth characteristic (this can be done manually or mobile app can do it). When this happen, the start bootloader flag will be set and system will be reset, then immediately after at boot it will start the bootloader.

wireless_remote: the same as for TSDZ2_wireless but here we need to save the power so Bluetooth is not always active. Maybe can be just the same process to activate it as currently for change ANT ID -- user will then be able to change ANT ID or start the bootloader.

What do you think? I hope it is now more clear, at least for me.

 

[rananna]

.
What do you think? I hope it is now more clear, at least for me.

Makes sense. I will start on it.

 

[casainho]

Makes sense. I will start on it.

Another thing, I was thinking on the encryption and while I was thinking it should be disabled, I don't think it is a good idea otherwise anyone would be able to start the bootloader and flash a buggy / not working firmware on purpose, rendering the EBike useless!!

A good idea can be to keep encryption on bootloader enabled and our keys will be public on our repository source files. If a user want to be secure, then this user will need to change to some random custom encryption keys for his own and then build both the bootloader and the firmware with that custom encryption keys.

 

[rananna]

A good idea can be to keep encryption on bootloader enabled and our keys will be public on our repository source files. If a user want to be secure, then this user will need to change to some random custom encryption keys for his own and then build both the bootloader and the firmware with that custom encryption keys.

Good call. I will do this.

 

[stancecoke]

My Flutter app is now working in principle.
It works with a simple HM10 BLE Module or with the commercial Lishui BT-Module

I'm now wondering, how to implement the ANT+LEV syntax.
Of course it's not really senseful to send an ANT+ message via UART, but as written before, I do not want to invent the next "homebrewed" protocol.
The message has to have this general ANT-structure

Sync - MsgLength - MsgID - MsgContent - Checksum

The MsgID should be 0x4E for "Broadcast Data" with the 8byte payload of the LEV pages?! The Controller would send one page every 250ms in the given pattern of 4 pages

DataPage#1 - DataPage#2/34 - DataPage#3 - DataPage#variable

Page#16 would be sent from the app to the controller for setting the assistlevel etc.

Pages 6-15 are reserved for future use, I wonder if they could be used for project specific data like parameter settings of the motor controller.... Is there a page space reserved for device specific functions?!

regards
stancecoke

 

[casainho]

My Flutter app is now working in principle.

Nice aspect!! I tried to run / debug on Code Studio but I got some errors... which IDE are you using?

And I would like to know the efforts and possible costs to make available this app for IPhone - does anyone know??


On my side, I am using 2 characteristics only (like if having two serial ports): one for send and receive periodic / real time data and other for send / receive the configurations. Later I hope we can implement the JSON as also the specific Bluetooth cycling profiles.

I now have configurations reading correctly from the wireless board but when I write them back when clicking on the OK button, the firmware of the wireless board crashes. Seems to is related to Bluetooth specification of writing max of 20 bytes at a time... but I am now looking at sample code from Nordic that let´s transfer up to 512 bytes at a time...

 

[stancecoke]

I tried to run / debug on Code Studio but I got some errors

have you run flutter pub get?

which IDE are you using?

I'm using Android Studio. I don't know how it works with Linux...

does anyone know??

I don't know any IOS developer... :?

I started to write the LEV-pages today. Page#1 and Page#16 is available already
https://github.com/EBiCS/EBiCS-Flutter/blob/master/lib/LEV_Pages.dart

regards
stancecoke

 

[casainho]

Configurations are working!! I solved the issue of writing max of 20 bytes at a time and now max writing value is 250 bytes, more than enough for the full configurations.

Next step is implement the full configurations and that will be a work of copy-paste of the ones that are already working as seen on this video:

[youtube]RCztvzQ0O6A[/youtube]

 

[stancecoke]

max writing value is 250 bytes, more than enough for the full configurations.

Hm, so you are inventing something "homebrewed" again . I had the hope, that we can create a "standard" for the motor settings, too. In analogy to the LEV-Standard, define pages with 8 byte payload and send them in the ANT-Syntax "Sync - MsgLength - MsgID - MsgContent - Checksum"
Some pages could be project specific, but I think most parameters will be similar in all firmware-projects...

regards
stancecoke

 

[casainho]

max writing value is 250 bytes, more than enough for the full configurations.

Hm, so you are inventing something "homebrewed" again . I had the hope, that we can create a "standard" for the motor settings, too. In analogy to the LEV-Standard, define pages with 8 byte payload and send them in the ANT-Syntax "Sync - MsgLength - MsgID - MsgContent - Checksum"
Some pages could be project specific, but I think most parameters will be similar in all firmware-projects...

The way Bluetooth works, I don't need all that complexity you mention - seems you need it because you are using only serial over Bluetooth. We will not use that as we don't have that limitations and in fact we will implement the Bluetooth cycling specific profiles that again you can't.

I want to develop fast so I am going to do the least work possible.

Even the Android app, I hope in future to be a better one in Flutter for Android and Iphone. For now, this one works and is the one I have knowledge to develop, this way later I can easier move to other.

I hope you can implement graphs, that is the only feature this Android app has that yours don't.

If later we decide to use your app, then we can see how we can make the connection to it. We have all the flexibility on our firmware side.

 

[stancecoke]

I don't need all that complexity you mention

That's not complex but easy. It has nothing to do with data lenght, the HM10 module handles messages with more than 20 bytes without problems. It's just a matter of standardisation and flexibility.
If you put the parameters into small data packets, it is much easier to maintain, adding new pages or changes in single pages do not disturb existing, established pages at all. You can work wonderfully with DMA, because the packet size is always the same....

Just my 2 cents...

regards
stancecoke

 

[casainho]

I don't need all that complexity you mention

That's not complex but easy. It has nothing to do with data lenght, the HM10 module handles messages with more than 20 bytes without problems. It's just a matter of standardisation and flexibility.
If you put the parameters into small data packets, it is much easier to maintain, adding new pages or changes in single pages do not disturb existing, established pages at all. You can work wonderfully with DMA, because the packet size is always the same....

Before of you, @raw did propose a Flutter app and JSON file for configuration. For me seems the best idea to use the JSON - would that be ok for you? -- read here the original message:

quote=raw post_id=1560232 time=1590840173 user_id=72166
@casainho

thank you for sharing info about your setup. i am now thinking about upgrading my watch

i have finished my concept of the configuration screen and JSON structure. i expect it to change a bit over time, but now we have at least something we can discuss about.

here is my draft of how the configuration that is sent from the dongle to the app looks like:

{
  "version": 0.1,
  "config": [
    {
      "type": "bool",
      "label": "boolean option",
      "key": "enable_walkmode",
      "value": false
    },
    {
      "type": "menu",
      "label": "Some submenu",
      "key": "some_submenu",
      "children": [
        {
          "type": "text",
          "label": "here you can type some informative text"
        },
        {
          "type": "bool",
          "label": "Enable Walkmode",
          "key": "enable_walkmode",
          "value": false
        },
        {
          "type": "title",
          "label": "powermode options"
        },
        {
          "type": "enum",
          "label": "Power Mode",
          "key": "power_mode",
          "value": "TORQUE",
          "choices": {
            "TORQUE": "by torque",
            "POWER": "by Power",
            "SLOPE": "by Slope"
          },
          "info": "Some description\nof the power modes\n\nRTFM!"
        }
      ]
    },
    {
      "type": "number",
      "label": "Maximum speed",
      "key": "max_speed",
      "value": 25,
      "min": 0,
      "max": 50,
      "step": 1
    },
    {
      "type": "number",
      "label": "something with two decimal places",
      "key": "some_decimal",
      "value": 2.4,
      "min": 0,
      "max": 5,
      "decimal": 2
    }
  ]
}

the configuration screen that gets generated from this json looks like this:

the "info" field can be added to any field type and is optional. i thought it would be nice if you could give some info for more complex options.

everything that has a value also has a key so we have something to reference when sending the configuration back to the dongle. i have not yet decided if the returning configuration values will be nested in the same tree or just one flat key:value object. for the key:value object, the keys need to be unique while for the nested stuff the submenus need a key too.

having a key on the submenu will also allow it to be translated later (a translation definition could override the labels based on the keys).

the enum type is currently only a string-enum. it would be possible to implement an int-enum but i was not sure if this makes something easier.

the number type can either be decimal or int (defined by the decimal field). currently, if it is decimal, the step field is not supported, it only works for int. min and max value is always required, step and decimal is optional.

while the configuration menu on the 860c is used for configuring the motor, the display and to show diagnostic values, the configuration menu expressed by this JSON is only used for dongle/motor configuration. if the app is started or the config screen is opened, the configuration will be fetched once. when the configuration is closed, the values will also be transfered to the motor once.

for diagnostic and frequently changing values (assist level) we will use different endpoints.
if you want to try youtself, here is the apk: https://easyupload.io/cgasmh the example config file is compiled in.

any comments or ideas on this?

my next step is to add bluetooth functionality and design the bluetooth api by adopting some Nordic NRF example with my dongle.
/quote

 

[stancecoke]

@raw did propose a Flutter app and JSON file for configuration.

Of course a JSON Message could be used, but I see some major disadvantages.
1. JSON knows no checksum, so invalid messages could cause very much trouble.
2. The JSON Message format carries a lot of overhead (names, qoutation marks, etc. as it is simply a string) that has to be sent over a slow, old fashiond UART interface.
3. which JSON parser do you recomment for the very limited computing power in the STM8 / STM32 world and the IDE's/compilers, we are using?

So I still think, using the ANT syntax is still a very good idea.

regards
stancecoke

 

[casainho]

Of course a JSON Message could be used, but I see some major disadvantages.
1. JSON knows no checksum, so invalid messages could cause very much trouble.

1. No checksum needed as that is done already on lower layer by Bluetooth. All data I send and receive to the Bluetooth characteristics is always correct, that is why I prefer to write and read directly to them. On ANT on our wireless board also, all data sent / received is always good, no need for checksum.

2. The JSON Message format carries a lot of overhead (names, qoutation marks, etc. as it is simply a string) that has to be sent over a slow, old fashiond UART interface.
3. which JSON parser do you recomment for the very limited computing power in the STM8 / STM32 world and the IDE's/compilers, we are using?

2. 3. The big flexibility we would have on the app configurations, results on the overhead of JSON.
Our wireless board should be ok with JSON and the software SDK from the manufacturer includes this JSON library: https://github.com/DaveGamble/cJSON

We have a very different hardware capabilities, you are trying to use a Blueooth <-> serial module and we are trying to use a programmable SOC with ANT and Bluetooth, that gives a lot of flexibility because of being programmable with a lot of processing power and memory.

The idea of using standards is mainly because of the value we can get. On the case of ANT, we get displays of different brands with WAY MORE features we could ever develop for EBikes, like the Garmin Edge with GPS navigation, specific cycling fitness metrics, etc.

What you offer is a possibility for us to use the mobile app you are developing and that would be an advantage to define a standard that we could then also use. The immediate value for us would be the not need to develop the app - but if your app limit us, do not work on my phone, do not work for Iphone and has no graphs, then I don´t see much value compared to the one we already have based on the ESP32-TSDZ2 project. Still, I think it is important if we could join our efforts and use the same app. For now, I think you should move ahead and make your system work, later we can see how we can interface our wireless board and maybe will be the way your are developing. For me is to early, I can´t do many things at the same time, for now I am trying to have the current Android app working at the same time I develop the firmware. When I have something that works, that I will make a release, then I should think in all this and see -- I tested the @raw Flutter app and I saw that Bluetooth does not work on my phone so I am not in a hurry to use an app based on Flutter.

 

[stancecoke]

No checksum needed as that is done already on lower layer by Bluetooth.

Yes, of course, but I'm talking about the UART communication with the motor-controller, that will be necessary with your nRF52840 or ESP32 boards also. This could become our lowest common denominator, I think.
But sadly I see, we won't find that denominator. :-(
Good luck with your project!

regards
stancecoke

 

[casainho]

Yes, of course, but I'm talking about the UART communication with the motor-controller, that will be necessary with your nRF52840 or ESP32 boards also. This could become our lowest common denominator, I think.

Ok, on the motor controller communications side, I already have a solution implemented that works and took me a lot of time to get it - I will not change it now because it works and I am very busy with the other developments.

On the wireless board communications to the mobile app, I hope in the future to use the JSON on top of the read/write directly on the Bluetooth characteristics. Also I hope to implement the specific cycling profiles of Bluetooth. For now, I have no JSON but I am already reading/writing directly on the Bluetooth characteristics, the same as used on the ESP32-TSDZ2 Android app, and that is why I am doing like this because it was fast to have it working.

I think the common technology can the be mobile app. Once I have the firmware developed and working with current Android app, then I can think on the next steps for the mobile app. Although I know that a Flutter app using Bluetooth currently will not work on my Samsung Phone (and so possible on other phones as well), I hope that it will be solved. I am being learning a bit a about Flutter because I want to use in future in another project so I think will be possible for me to collaborate in your app or on the @raw app. I will follow your development!!

 

[casainho]

Wireless firmware update bootloader is ready!!

@rananna did the work and the bootloader hex files, sources and technical description are here: https://github.com/OpenSource-EBike-firmware/TSDZ2_wireless-bootloader

@rananna also did a pull request for the wireless remote support this wireless firmware update bootloader: https://github.com/OpenSource-EBike-firmware/ebike_wireless_remote

To resume:
- firmware can be updated using Bluetooth connection on a smartphone or a PC
- firmware can be updated using USB on a PC (requires physical connection, maybe this option will not be the favorite)

To start the bootloader / firmware upgrade, user will need to do one of the following options:
- long press the button in the TSDZ2 wireless board at startup (on the remote wireless, long press all buttons)
- use NRFconnect on smartphone or PC to write 1 to Bluetooth specific characteristic for enable the bootloader

Here is a video @rananna did explaining how to update firmware on a PC and using the Blueooth connection:
[youtube]E0GXf4Y68pM[/youtube]

 

[rananna]

@rananna also did a pull request for the wireless remote support this wireless firmware update

I would also like to add rhe bootloader bluetooth characteristic to the TSDZ2_wireless firm?are. Would you mind if I made the mods and submit a pull request?

 

[casainho]

@rananna also did a pull request for the wireless remote support this wireless firmware update

I would also like to add rhe bootloader bluetooth characteristic to the TSDZ2_wireless firm?are. Would you mind if I made the mods and submit a pull request?

That would be nice - I just updated the master branch with my latest code where the configurations done by mobile app are working. To be clear, do the pull request against the master branch.

When you do it, I will be able to test the bootloader and I will give feedback.

 

[rananna]

@rananna also did a pull request for the wireless remote support this wireless firmware update

I would also like to add rhe bootloader bluetooth characteristic to the TSDZ2_wireless firm?are. Would you mind if I made the mods and submit a pull request?

That would be nice - I just updated the master branch with my latest code where the configurations done by mobile app are working. To be clear, do the pull request against the master branch.

When you do it, I will be able to test the bootloader and I will give feedback.

Ok, I will make the mods and do a pull request again the master branch.