Vida CEM swapping

[dikidera]

I fixed it, the issue was..my program was not being fully uploaded because of a bug in the code.

Afterwards the code works. Next I encountered an issue with transmission quality, I would get a CAN error frame, but it wasn't the ECU's fault, somehow my SPI1.0 CAN device was breaking the frames, and I was losing up to 1000 bytes, my other CAN controller on SPI0.1 was fine with the same speeds. So either I have a faulty CAN controller or the RPI4's SPI1 bus somehow corrupts the data.

I was almost worried I have to reduce the transmission speed.

The next issue is duplicate frames, the SBL sends exactly 262144 bytes, but I receive 40-80 bytes more.

[imransterling]

Screenshot_20180616-162942.png

Bro you can do this job for about $70- hardware , software included. This youtube video and its seller will provide you all that.

[5ft24]

I fixed it, the issue was..my program was not being fully uploaded because of a bug in the code.

Afterwards the code works. Next I encountered an issue with transmission quality, I would get a CAN error frame, but it wasn't the ECU's fault, somehow my SPI1.0 CAN device was breaking the frames, and I was losing up to 1000 bytes, my other CAN controller on SPI0.1 was fine with the same speeds. So either I have a faulty CAN controller or the RPI4's SPI1 bus somehow corrupts the data.

I was almost worried I have to reduce the transmission speed.

The next issue is duplicate frames, the SBL sends exactly 262144 bytes, but I receive 40-80 bytes more.

The RPI's SPI bus clock is based off the SOC clock. The RPI varies that clock depending on CPU load. That's why you don't see many RPI's being used for timing critical applications, but Teensy and Arduino are. I built a home power monitoring system using a Pi. Uses the SPI to read the DAC on the hat to read the 4 current transformers. Even though the pi supports high SPI clock rates, it was giving dropouts in the data. Fix was to Set Turbo on and set the governor to performance so it runs at a fixed clockrate. Even then, I had to go to a 4.x version kernel. 5.x has a lot of other timing quirks that hose up SPI

[dikidera]

Thanks, I will try applying your advices to my kernel. I also have some information about this too, the MCP2515 controller is ubiquitous, but is not perfect and not really enough for high-speed applications. So the SPI bus and kernel stuff are one part of the problem. Additionally, the latest kernel 5.15 ships with a change in the mcp251x driver that causes the driver to hang on mcp251x_hw_wake, so it had to be recompiled to mcp251x_hw_reset instead.

And here are my findings.

The reason I was getting more bytes, was because the SH7055 MCU was silently re transmitting messages that were deemed broken by it or the CAN protocol. They are fine on my end, but they are nevertheless sent again automatically by the MCU without any indication. I did a small hack where I flip a bit in the EXTENDED_ID, for one message and flip it again for the next, alternating between 1 and 0 to filter out duplicates which seems to work. This will of course not detect entirely absent/skipped messages.

On one of my can controllers I got many RX errors, which is not nice, while the other only got 1. It's either a driver issue, a SPI bus issue, or my second CAN controller is bad.

Of course this is also caused by the design of the SBL which rather than working on a event/command loop, spams the CAN bus with the memory contents with no user-controllable way to retransmit data or verify I actually go it, but it is fast nevertheless.

In the meantime, here is a partially rewritten uploader of the SBL.

Code: Select all

# import the library
import can
import time
import sys
import asyncio

tSleep = 0.01
address = 0xFFFFA000 #HARDCODED UPDATE
cmd_set_pointer_template = [0x7A, 0x9C, 0x00, 0x00, 0x00, 0x00]
cmd_write_data_template = [0x7A, 0xAE, 0x00, 0x00, 0x00, 0x00]
cmd_set_pointer_address = [0x7A, 0x9C, 0xFF, 0xFF, 0xA0, 0x00]
cmd_launch_sbl = [0x7A,0xA0]
if len(sys.argv) > 2:
    binFile = sys.argv[2]
else:
    binFile = 'main2.elf'

bus = can.ThreadSafeBus(interface='socketcan',
                  channel=sys.argv[1] if len(sys.argv) > 1 else 'can0',
                  receive_own_messages=False)
    
def send_sleep_cmd():
    starttime = time.time()
    
    while True:
        if time.time() - starttime >= 1:
            break
            
        message = can.Message(arbitration_id=0x000ffffe, is_extended_id=True,
                                  data=[0xFF, 0x86, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00])
        #print(message)                      
        bus.send(message)
        time.sleep(tSleep) # we consume all of the MCP write buffer too fast if we dont sleep
        
def send_can_message(msg, arbid=0x000ffffe, print_msg=False):
    message = can.Message(arbitration_id=arbid, is_extended_id=True,
                          data=msg)
    #if print_msg:
    print(message)
    bus.send(message)
    
def save_to_file(fileName, fileData):
    f = open(fileName, 'wb')
    f.write(fileData)
    f.flush()
    f.close()
    
def read_file_contents():
    f = open(binFile, 'rb')
    fileData = f.read()
    f.close()
    
    return fileData

    
async def can_listen():
    bytes_received = 0
    buffer = bytearray()
    prevArbid = None
    while True:
        message = bus.recv(1.0)  # Timeout of the SBL. If we dont receive a message, we probably already sent everything

        if message is not None:
            if message.arbitration_id == 0x0000FFFE or message.arbitration_id == 0x0004FFFE:
                if prevArbid == message.arbitration_id:
                    print("dup?")
                    continue

                bytes_received += 8
                buffer += message.data
                prevArbid = message.arbitration_id
                
                message = None
            else:
                print(message)
        else:
            break
    print(bytes_received)
    save_to_file('dump.bin', buffer)
    
    


async def main() -> None:
    global address, cmd_set_pointer_template, cmd_write_data_template
    global cmd_set_pointer_address, cmd_launch_sbl


    # send a message
    send_sleep_cmd()

    #sleep to ensure ECU got this message. 

    #time.sleep(1)



    computedAddress = bytearray(address.to_bytes(4, 'big'))

    cmd_set_pointer_template[2] = computedAddress[0]
    cmd_set_pointer_template[3] = computedAddress[1]
    cmd_set_pointer_template[4] = computedAddress[2]
    cmd_set_pointer_template[5] = computedAddress[3]

    send_can_message(cmd_set_pointer_template)
    time.sleep(1)
    

    fileData = read_file_contents()
    fileLen = len(fileData)
    #print("File len {}", fileLen)
    filePtr = 0


    cmd_write_data_template[2] = fileData[0]
    cmd_write_data_template[3] = fileData[1]
    cmd_write_data_template[4] = fileData[2]
    cmd_write_data_template[5] = fileData[3]

    send_can_message(cmd_write_data_template)
    
    filePtr += 4
    address += 4
    bytes_left = 4
    for msg in bus:
        print(msg)
        
        if filePtr >= fileLen:
            bytes_left = filePtr - fileLen
            
            if bytes_left == 0:
                break
                
        if msg.arbitration_id == 0x00000021:
           
            #print(msg.data[2:])    
            if msg.data[1] != 0x9C:
                continue

            computedAddress =  bytearray(address.to_bytes(4, 'big'))
            cmd_set_pointer_template[2] = computedAddress[0]
            cmd_set_pointer_template[3] = computedAddress[1]
            cmd_set_pointer_template[4] = computedAddress[2]
            cmd_set_pointer_template[5] = computedAddress[3]

            send_can_message(cmd_set_pointer_template)
            time.sleep(tSleep)
            
            # the code below simplifies the logic a bit, but may rewrite in the future
            if bytes_left == 4:
                cmd_write_data_template[2] = fileData[filePtr]
                cmd_write_data_template[3] = fileData[filePtr+1]
                cmd_write_data_template[4] = fileData[filePtr+2]
                cmd_write_data_template[5] = fileData[filePtr+3]
            else:
                cmd_write_data_template[3] = 0
                cmd_write_data_template[4] = 0
                cmd_write_data_template[5] = 0
                if bytes_left == 1:
                    cmd_write_data_template[2] = fileData[filePtr]
                elif bytes_left == 2:
                    cmd_write_data_template[2] = fileData[filePtr]
                    cmd_write_data_template[3] = fileData[filePtr+1]                
                elif bytes_left == 3:
                    cmd_write_data_template[2] = fileData[filePtr]
                    cmd_write_data_template[3] = fileData[filePtr+1]
                    cmd_write_data_template[4] = fileData[filePtr+2]
                
            send_can_message(cmd_write_data_template)
            time.sleep(tSleep)
            
                            
            address += bytes_left
            filePtr += bytes_left
                
        #print(hex(address - 2) + ': ' + hex(msg.data[2]))  

    send_can_message(cmd_set_pointer_address)

    time.sleep(tSleep)
    
    #start async task now so that the we dont timeout and then launch SBL
    task = asyncio.create_task(can_listen())
    
    send_can_message(cmd_launch_sbl)
    
    await task


if __name__ == "__main__":
    asyncio.run(main())

And here is the actual SBL, you may notice it's small but it definitely took time to ensure this works. It uses npkern as a base, it has many useful helper functions, the only thing missing here is the linker script which I had to modify slightly.

Code: Select all

#include "stypes.h"
#include "ivect.h"
#include "functions.h"    //for set_imask etc
#include "reg_defines/7055_350nm.h"

#define IVT_ENTRIES 0xA5
#define ARRAY_SIZE(x)    (sizeof(x) / sizeof((x)[0]))
#define WAITN_TCYCLE 4        /* clock cycles per loop, see asm */
#define WAITN_CALCN(usec) (((usec) * 20 / WAITN_TCYCLE) + 1)
#define WAITN_TSE_CALCN(usec) (((usec) * CPUFREQ / ATUPRESCALER))

#define EXT_FORMAT 0x00000008
#define DUP_BIT    0x00000020
#define END_BIT    0x00000040


/*$ sh-elf-gcc -c -m2 -mb  -Os -ffunction-sections -Dvolvo -fomit-frame-pointer -std=gnu99 -Wall -Wextra -Wstrict-prototypes -fstack-usage -fverbose-asm -Wa,-ahlms=main2.lst  -D SH7055 -D PLATF=\"SH7055\" -I . main2.c
sh-elf-gcc  main2.o -m2 -mb -nostartfiles -T./ldscripts/lkr_7055_Volvo.ld -Wl,-Map=main2.map,--cref,--gc-sections -o main2.elf
sh-elf-objcopy -O binary -S  main2.elf main2.bin   

$ sh-elf-gcc -c -m2 -mb  -Os -ffunction-sections -Dvolvo -fomit-frame-pointer -std=gnu99 -Wall -Wextra -Wstrict-prototypes -fstack-usage -fverbose-asm -Wa,-ahlms=main2.lst  -D SH7055 -D PLATF=\"SH7055\" -I . main2.c;sh-elf-gcc  main2.o -m2 -mb -nostartfiles -T./ldscripts/lkr_7055_Volvo.ld -Wl,-Map=main2.map,--cref,--gc-sections -o main2.elf;sh-elf-objcopy -O binary -S  main2.elf main2.bin

*/


static __attribute__ ((noinline)) void can_send(volatile uint32_t data1, volatile uint32_t data2);
static inline void setup_mailbox(void);
static void waitn(unsigned loops);

static void waitn(unsigned loops) {
    u32 tmp;
    asm volatile ("0: dt %0":"=r"(tmp):"0"(loops):"cc");
    asm volatile ("bf 0b");
}

static inline void setup_mailbox()
{
              
    //HCAN0.MC[15][6] = (*(unsigned char*)(start_address));  // docs MCx[8]      
    //HCAN0.MC[15][7] = 0xff;  // docs MCx[7]
    HCAN0.MBIMR.BIT.MB15 = 1;
    
    HCAN0.MBCR.BIT.MB15 = 0;
    HCAN0.MC[15][0] = 0xF; // DLC8  // docs MCx[1]
    
    //-----------
    
    HCAN0.MC[15][1] = 0; // docs MCx[2]
    HCAN0.MC[15][2] = 0;  // docs MCx[3]
    HCAN0.MC[15][3] = 0;  // docs MCx[4]
    //-------------
    
    
    
    HCAN0.MC[15][4] = EXT_FORMAT; // docs MCx[5]
    
    HCAN0.MC[15][5] = 0x0; //docs MCx[6]
    
    //fine
    HCAN0.MC[15][6] = 0xfe;  // docs MCx[7]
    HCAN0.MC[15][7] = 0xff;  // docs MCx[8]
}

static __attribute__ ((noinline)) void can_send(uint32_t data1, uint32_t data2)
{
    volatile uint32_t *ptr = (uint32_t *)&HCAN0.MD[15][0];
    volatile uint32_t *ptr2 = (uint32_t *)&HCAN0.MD[15][4];
    
    *ptr = data1;
    *ptr2 = data2;
    PB.DR.WORD ^= 0x8000;

    HCAN0.TXPR.BIT.MB15 = 1;
    PB.DR.WORD ^= 0x8000;
    while(HCAN0.TXPR.BIT.MB15)
    {
        PB.DR.WORD ^= 0x8000;
    }
        
    HCAN0.TXACK.BIT.MB15 = 1;
}

void main(void) {
   // UBC.UBCR.BIT.UBID = 0;
    //UBC.UBARH.WORD = 0xFFFF;
    //UBC.UBARL.WORD = 0xE400;
    //UBC.UBAMRH.WORD = 0x0;
    //MSTCR.WRITE = 0x3C0F;
    set_imask(0x0F);
    WDT.WRITE.RSTCSR = 0xA500;
    WDT.WRITE.RSTCSR = 0x5A00;
   // volatile int t = WDT.READ.TCSR.BIT.OVF;
    WDT.WRITE.TCSR = 0;
    
    setup_mailbox();

    
    volatile uint32_t start_address = 0x200000;
    volatile int size =  0x40000;
    volatile int count = 0;
    volatile char tec = HCAN0.TEC;
    while(count < size)
    {
        PB.DR.WORD ^= 0x8000;
          
        uint32_t data1 = *(uint32_t*)(start_address);
        uint32_t data2 = *((uint32_t*)(start_address + 4));
        
        can_send(data1, data2);
        
        HCAN0.MC[15][4] ^= DUP_BIT;

        waitn(WAITN_CALCN(100));
        
        start_address += 8;
        count += 8;

        //(*(unsigned int*)(0xFFFFA200)) = 0xdeadbeef;

    
    }
    HCAN0.MC[15][4] |= END_BIT;
    can_send(0x11111111, 0xbaadb00b); // trigger meaning we are done;
    
}

It's dead simple, but very fast. May not be absolutely reliable, but you get a complete dump in as little as 15 seconds@250kbits with 100 microsecond delay.

A further improvement in speed can be achieved by utilizing the 29 additional bits for storing more data, this would mean 11 bytes per CAN message.
Or you can trade this speed for more reliability by storing the address/pointer in the 24 bits, which would give you more information on duplicate frames as well as knowing where in the dump you are.

[rkam]

Well done.

I just tried your sbl (main2.bin) converted to vbf and uploaded with DiCE while logging the can channel with Kvaser.
I managed to get 32700 frames which should be something like 261600 bytes, so I have lost some.
Kvaser is supposed to manage 8000 frames per second, so I'm not sure what happened.
Receiving took 35.8656 seconds.

[dikidera]

It could be any number of reasons. Does the dump contain the very first bytes? I was thinking of adding a 1-2 second delay before I transmit CAN frames to ensure that the software/threads have setup the receive code by then. But you said you had a can logger.

In any case, the code only sends frames and it even has a 100 microsecond(0.1ms) delay. I am guessing that somewhere along the line, the CAN protocol is determining the message to be invalid and discarding, aka the Data Link Layer.

[rkam]

Yes, the first 0x8000 is all received well. I'll have to convert the log file to a binary dump to check the rest.
I got one frame error somewhere in the middle.

[dikidera]

Well I have found a pattern for the problem.

If the data to be sent is different from the previous data, the error is more likely to occur. For instance your 0x0-0x8000 data should be all 0xFF bytes, they are all the same, but as soon as the data starts to change, the error is more likely to occur. I have verified it a few times already.

The more entropy the data has, the more likely to have an error frame or duplicate. I tried all sorts of sleeps between transmissions(even serious 1 second pauses) and nothing managed to stop them.

0xFF 8 times is all bits set to 1, I am guessing something is happening at the OSI stack where the bits are more likely to be affected?

Addendum:

Haha, I was right!!!

can_send(0xFFFFFFFF, 0xFFFFFFFF); at max speed with no sleeps = no duplicate frames. Lets see how many CHANGING bytes I can send before duplicates and errors appear.

The answers seems to be, unspecified. I get better performance when they are 8 bytes per frame, but the bit order seems to be important. Something seems to be breaking at the protocol level.

[prometey1982]

@dikidera I didn't find checking for send completeness in your SBL. For example code from TF80:

Code: Select all

ROM:00000758     loc_758:                                ; CODE XREF: CAN1_sendMessage_to_1_mailbox_6FE+62j
ROM:00000758 018                 bsr     CAN1_checkMailbox1transmisionCompleted ; Branch to Subroutine
ROM:0000075A 018                 nop                     ; No Operation
ROM:0000075C 018                 extu.b  r0, r0          ; Extend as Unsigned (Byte)
ROM:0000075E 018                 tst     r0, r0          ; Test Logical
ROM:00000760 018                 bt      loc_758         ; check while message is writing to CAN1 1 mailbox

function:

Code: Select all

ROM:0000089E     CAN1_checkMailbox1transmisionCompleted:
ROM:0000089E                                             ; CODE XREF: CAN1_sendMessage_to_1_mailbox_6FE:loc_758p
ROM:0000089E 000                 mov.l   r14, @-r15      ; Move Long Data
ROM:000008A0 004                 sts.l   pr, @-r15       ; Store System Register Long
ROM:000008A2 008                 mov.w   #HCAN1_TXACK_W, r4 ; Move Immediate Word Data
ROM:000008A4 008                 bsr     readWordFromCAN1reg ; r4 - reg addr
ROM:000008A4                                             ; r0 - result
ROM:000008A6 008                 mov     #0, r14         ; Move Immediate Byte Data
ROM:000008A8 008                 mov.w   #b'1000000000, r2 ; Move Immediate Word Data
ROM:000008AA 008                 extu.w  r0, r4          ; Extend as Unsigned (Word)
ROM:000008AC 008                 tst     r2, r4          ; check 9 bit of CAN1 TXACT register
ROM:000008AE 008                 bt      loc_8B2         ; Branch if True
ROM:000008B0 008                 mov     #1, r14         ; Move Immediate Byte Data
ROM:000008B2
ROM:000008B2     loc_8B2:                                ; CODE XREF: CAN1_checkMailbox1transmisionCompleted+10j
ROM:000008B2 008                 mov     r14, r0         ; Move Data
ROM:000008B4 008                 lds.l   @r15+, pr       ; Load to System Register Long
ROM:000008B6 004                 rts                     ; Return from Subroutine
ROM:000008B8 004                 mov.l   @r15+, r14      ; Move Long Data
ROM:000008B8     ; End of function CAN1_checkMailbox1transmisionCompleted

[dikidera]

I do check, but I also went and found some Renesas documents and changed the code a bit, I was checking for TXPR but it in fact should be TXACK. Still it didn't change anything for transmission quality. Something happens at the protocol level and the only clue is the TEC register. But the 7055 mcu retransmits frames automatically.

Even with a professional tool it seems there are still problems as per rkam's post. So I know I am not the only one with the same problem. And my experiments show that if I send different data, the problem is more likely to occur.