Who's online and Who said what?

Ask any questions about using PicoScope Automotive software here.
Post Reply
Steve Smith
Pico Staff Member
Pico Staff Member
Posts: 740
Joined: Sun Aug 25, 2013 7:22 am

Who's online and Who said what?

Post by Steve Smith » Fri Feb 01, 2019 4:35 pm

One question that is continually raised and with CAN is “Who is online & who said what?

As we know, decoding CAN networks into useful, readable and valid information is challenging. topic21593.html?&p=97888#p97888

Let’s start right at the beginning with a snapshot of CAN traffic.

Here we are able to capture, then evaluate the integrity of the CAN signals (Physical Layer).

The speed and sample rate of the scope allows us to look at CAN signals as though we were looking through a microscope.

Below we have a glimpse of 2 CAN packets travelling through a twisted pair of CAN BUS wires, revealing the correct upper and lower voltage levels (3.5 V CAN Hi & 1.5 V CAN Lo) along with the correct “idle” voltage of 2.5 V
IMAGE 1.png
Image 1
Note how the voltage “ringing” and “over-shoot” below is typical of automotive CAN signals and can be attributed to a number of factors such as the style of test lead (standard or high impedance) environmental noise, and “end of message” confirmation.
IMAGE 2.png
Image 2
Remember that none of these are faults and CAN networks are immensely fault tolerant when it comes to the Physical Layer. Here is a typical example topic20981.html?&p=93851&hilit=fault+tolerant#p93851

If the above capture is to be considered “looking through a microscope” at the flowing CAN traffic, then the capture below should be compared to looking at flowing CAN traffic from a distance, but with the option to view in greater detail thanks to a pair of binoculars (referred to as zoom)
IMAGE 3.png
Image 3
To put this volume of CAN data into perspective, the time across the screen is just 1 second (100 ms/div x 10 div) and using the CAN decode feature of PicoScope we conclude a total of 1066 message in just 1 second of passing traffic!

To display the decoded CAN data take a look at the image below.
IMAGE 4.png
Image 4
If you are not sure of the “Baud Rate” (network speed) try this technique below.
IMAGE 4B.png
Image 4B
Note how the decoded example above presents a warning to the user that the “sample rate may be too low” (the properties window indicates 1 MS/s where the ideal would be 2 MS/s min) which consequently may lead to erroneous CAN frames highlighted in red within the decode table.

To avoid this scenario we can use a number of software and hardware features to ensure a superior sample rate (sufficient to decode high speed CAN)

There are a number of decoding techniques listed here:
https://www.picoauto.com/library/traini ... onus-class

Focusing further on CAN decoding, how can we ensure a sufficient sample rate to provide efficient serial decoding of high speed CAN, whilst allowing a practical time span on the screen to assist with CAN controller identification? Remember the “end game” here is to reveal the ID’s associated with a number of a number of CAN controllers.

The first tip is to increase the time span across the screen and the number of selected samples. Below I have used 500 ms/div (total time across the screen 5 seconds) with 10 MS which delivers 2 MS/s to ensure the integrity of our decoded CAN data.

10,000,000 Samples across the screen / 5 seconds screen time = 2,000,000 S/s

Next, rather than consume 2 x channels with CAN data, if we are happy with the formation of our CAN signal (see image 1 in this post) use channel A only to capture the CAN Hi signal via a High Impedance Probe (TA375) to minimize noise, ringing and signal intrusion. (Protect the integrity of the CAN signal)

More information can be found here on the TA375 High Impedance probe topic21591.html?&p=97872#p97872

The capture below highlights how the software settings and chosen hardware has a positive effect upon the capture of our CAN Hi network signal. Whilst the software still warns the user of a possible sample rate concern, we can see that every message in the decode table is without error. Note also how we now have a total of 5318 CAN frames to wade through!
IMAGE 6.png
Image 6
The vehicle in question above is a 2013 BMW 2.0 Diesel F31 Auto where by the shift lever indicates the gear position via the high speed CAN network. (The gear lever is in effect a CAN Controller)

Channel B uses the TA234 current clamp https://www.picoauto.com/products/curre ... rent-clamp attached around the shift lever CAN Hi branch wire to act as a “location finder” for any transmissions sent from the shift lever.

To clarify the above, where we have a voltage differential (CAN Hi - CAN Lo = 0 V -2 V) there will be current flow (CAN Bus load approx. 60 Ohms) from the shift lever when messages are transmitted onto the CAN Bus network. Note the sequential spikes from the current clamp!

Using the Zoom, Time Rulers and Scaling features, we can measure the time span between each spike captured by the current clamp, approx. 30 ms. If we then place the mouse pointer over the decode bar (at either Time Ruler/current peak) within the PicoScope display (graph) a pop up window will reveal information about the CAN packets within the area of the mouse pointer.
IMAGE 7.png
Image 7
This technique is very dependent on how steady you are with the mouse pointer and so I would recommend zooming directly above one of the Time Rulers and increasing the scale of channel B (x 50) to align the current spikes (Channel B) with each CAN packet (Channel A)

Notice how the peak current flow (within the shift lever CAN Hi branch wire) always coincides with a transmission from a CAN Controller with the ID 1 97 and these transmissions occur every 30 ms!
IMAGE 8.png
Image 8
This is starting to look like the shift lever has the CAN ID 1 97!

Could we therefore categorically determine not only the CAN ID’s of the Shift Lever but other CAN Controllers using this same technique?

Before we move on there a few pointers to be aware with CAN ID’s:

The CAN ID serves a number of purposes, it informs of 11 or 29 bit CAN and the priority of the pay load (Data) The lower the numerical ID the higher the message priority.
The same CAN Controller can also produce multiple ID’s.

For example, engine coolant temperature ID maybe be grouped in “Temperature Related” messages, where the engine speed ID could be grouped with “Engine related” messages. Both ID’s have come from the same CAN Controller

A clue to the identification (ID) of a CAN Controller can be found in the “Arbitration” given arbitration is won based on message priority.
E.g. If we discover the byte within the “Data Field” that denotes the “Brakes ON or OFF” message, we could assume this data most probably originated from the ABS CAN Controller and hence we have now identified the ABS CAN Controller ID. This is not an exact science as the “Brakes ON or OFF” message could also come from the Cruise Control CAN Controller!

Basically what we are saying is that don’t be surprised if you discover multiple ID’s originate from a single CAN Controller (ECU)

Returning now to CAN ID’s, I mentioned earlier how current flow occurs every 30 ms from the shift lever, lets correlate these transmissions (current flow) within the CAN decode table using the “Filter Function”

Within the decode table click on the “Filter” button and enter 197 above the ID column to list only the CAN packets relevant to ID 1 97

Note how the Start and End time of each transmission from CAN Controller ID 1 97 is identical to the spikes detected by the current clamp (30 ms intervals)
IMAGE 9.png
Image 9
I should have mentioned here that filtering the current clamp signal is going to be essential as we are pushing the limits of the 30 A current clamp when we are measuring approx. 10 mA

Looking at the Data field in the decode table (highlighted yellow) we can see how the first 3 bytes of data are constantly changing with no activation of the shift lever (not sure why)

If we were now to operate the shift lever we could look for a change in the data field (filtered column 197) that would help confirm we have correctly identified the shift lever as CAN ID 1 97

Now, time to take a breath:

PicoScope is not a CAN Decoder in the true sense of the word! We cannot collect continual streams of endless CAN data as we can with devices such as the Intrepid Vehicle Spy or X-Analyzer 3 loggers and software suites.

We (PicoScope) are decoding the Physical layer (BUS voltages) where these CAN Loggers (Intrepid etc.) are decoding the Silicone layer within the CAN controller. What we have on screen and in the Waveform Buffer is all we have to work with and so we have to get “clever” trying to identify CAN ID’s

An example of CAN decoding (reverse engineering) using dedicated CAN data loggers can be found here:
https://www.youtube.com/watch?v=Csn2pV3 ... e=youtu.be

Using a 3rd channel of PicoScope (Channel C) a pressure switch was placed on top of the shift lever “Park” switch in order to indicate within our displayed waveform when the Park switch was activated. An NVH interface (TA259) was used to provide power to a micro switch which in turn provided an output to PicoScope via the TA259 BNC output.

A couple of tips here, use this technique with the ignition on but engine off to minimize CAN activity and of course, ensure battery support is installed
IMAGE 10.png
Image 10
Returning now to the decode table select “Display Packets Between Rulers” and “Filter” for ID 197. Scroll down the decode table and look for a change in the data bytes relative to the operation of the Park switch
IMAGE 11.png
Image 11
Operating the “Park” switch forces a byte change from the shift lever on the CAN BUS where we can see the fourth data byte change from C0 to D5 (press Park) and return to C0 (release Park).

Given the shift lever is transmitting data every 30 ms, there is no change in the frequency of the current spikes. (I was expecting there to be an instant current spike when pressing the Park button) I guess 30 ms transmission intervals is sufficient to ensure any change in the shift lever status/operation is executed in a timely fashion.

All in all there is a lot to digest here with CAN ID’s and there are other techniques to demonstrate. One of which is CAN Controller/ECU disconnection topic21593.html?&p=97888#p97888

I will add to this post 2 other techniques to reveal CAN ID’s ASAP

To close for now, an understanding of the CAN network and structure you are diagnosing is essential and an accurate CAN topology or wiring diagram from the Vehicle Manufacturer is essential. These will reveal CAN “branches” and ensure the CAN line in which you are measuring current, is only capturing current from one source

Remember PicoScope is far more than an oscilloscope, amongst its numerous applications is a serial decode function. Whilst it is not a dedicated serial decoder we can manipulate the software to deliver decode/logger features. Knowing its strengths and weakness and applying the techniques above will help. (When it’s all you’ve got, then it’s all you’ve got!)

To summarize these techniques.

1. Use a single channel to obtain CAN Hi via a TA375 High Impedance Probe
2. Use 500 ms/div with 10 MS across the screen to obtain 3 buffers with 3 active channels
This equates to 15 seconds of data in total = approx. 15,000 CAN packets
3. Using 2 x monitors helps to position your decode table away from your PicoScope screen
4. Filter the current clamp (Low pass 1 kHz) use the zoom and scaling functions of PicoScope
5. Use the Time Rulers as markers/location finders in your captured CAN signal
6. Use the CAN Decode table Filter option for the ID or Data of choice
7. Measure current on a CAN branch wire only (Here we used shift lever CAN Hi)
8. Force a change of state from the CAN Controller (Here we pressed the Park switch)
9. Disconnect the CAN Controller of interest topic21593.html?&p=97888#p97888 Be careful of termination resisters, and “loop through” See Ben’s case study here: https://www.picoauto.com/library/case-s ... ions-fault
10. Force a change of state using an “Active Test” via a Scan Tool
E.g. Operate Cooling fan via scan tool and use cooling fan current as your guide to the CAN data of interest (location finder)
11. Use Excel which I will go through in the next post (As soon as I understand what I am doing)

I hope this helps, take care……Steve

Steve Smith
Pico Staff Member
Pico Staff Member
Posts: 740
Joined: Sun Aug 25, 2013 7:22 am

Re: Who's online and Who said what?

Post by Steve Smith » Fri Feb 15, 2019 10:35 am

Thanks for reading up to this point, here is a “follow-on” with additional information surrounding points 9 and 10:

9. Disconnect the CAN Controller of interest
10. Force a change of state using an “Active Test” via a Scan Tool

Before we continue ensure the ignition is on, engine is off & battery support installed.

Below we have the low pressure, fuel priming pump control unit disconnected from the CAN network. “Fuel Control” is missing from the vehicle “ECU search” via the diagnostic scan tool

Fuel Pump Controller Offline
IMAGE 1.png
FP Offline Scan
IMAGE 2.png
FP Offline Scope
Using PicoScope we obtain 2 captures of the CAN network, one with the Fuel Pump Controller disconnected (as above) and one with the Fuel Pump Controller connected (below)

Fuel Pump Controller Online
IMAGE 3.png
FP Online Scan
IMAGE 4.png
FP Online Scope
In both captures, decode the data for all buffers (see image 4 previous post) and Export this data to a location on your PC. (You will have 2 x .csv files saved to a location on your PC)

Now open your exported .csv file containing the data with all ECU’s online. (Fuel pump controller connected-FP ON LINE)

Click on column “E” so that the entire list of ID’s are highlighted > Right click and select “Copy”
Now click on column “Q”, Right click and select Paste. This will copy all ID’s captured whilst all ECU’s were on line and copy them to a new column (Q). At this stage it would be good practice to give column “Q” a new header title to avoid confusion with so many lines of data. I have renamed column “Q” header as “ID FP ON”.

To reduce the amount of data (within column “Q”) click on the “Data” tab and select “Remove Duplicates” which removes over 10,000 ID’s; followed by “Sort” A-Z to place the ID’s in column “Q” in a logical order
Next, open your exported .csv file containing the data with the ECU of interest disconnected. (In this case it was the Fuel Pump Controller – FP OFF LINE) Right click on column E and copy the data to column “R” of the “FP ON LINE” .csv file. Now repeat the process above for removing duplicate ID’s, placing the data in a logical order and renaming column “R” (I have used ID FP OFF)

Finally, click on the “Home” tab, highlight columns “Q & R” and select “Conditional Formatting” > Highlight Cells Rules > Duplicate Values and select Unique values.
This will then highlight a number of ID’s that were present when the fuel pump controller was online (Column Q) but missing when the fuel pump controller was disconnected (Column R) Copy and paste these highlighted values into column “S” as these become our unique ID’s of interest

IMAGE 5.png

Here we have identified ID’s 2 26, 2 F8, 3 35, 3 67, 4 4C, & 5 17 are all missing when the fuel pump controller is taken off line. The ID 0F FF BF B9 is a “curve ball” at present as this ID appeared when the fuel pump controller was disconnected! I will dig deeper on this one ASAP

The video link below will take you through this whole process.
https://drive.google.com/file/d/1K0P1z2 ... sp=sharing

Forcing the fuel pump to activate using the scan tool is the next step, whilst capturing fuel pump current, and CAN Hi data simultaneously in a single waveform buffer.
IMAGE 6.png
ID 3 35
Using the Decode Filter option enter each ID above the ID column to determine the relevance of the ID during fuel pump operation. Starting at the top, 2 26 & 2 F8 could not be found. ID 3 35 however appeared 4 times in 5 seconds at approx. 1 second intervals (look at “Start Time”). Interestingly enough, the first transmission of ID 3 35 was 50 ms before the in-rush current into the fuel pump. Note also how the “Data” column indicates a change in the information carried in the payload, suggesting an information update is taking place.

Moving on, ID 5 17 was the only other ID that was present during fuel pump operation, with transmissions every 646 ms approx. Note how the data field does not change during fuel pump operation, suggesting the data is not relevant to fuel pump operation. (This is an assumption at best and therefore cannot be discounted 100%)
IMAGE 7.png
ID 5 17
Hopefully we are slowly narrowing down multiple ID’s to just two possibilities (3 35 & 5 17)

Using the “Decode Between Rulers” option we can begin to pinpoint the transmissions of CAN ID’s relative to fuel pump activation, so allowing for judgement on the relevance of ID’s 3 35 & 5 17
IMAGE 8.png
Decode between rulers
Above we can see how we are reducing the CAN data even further thanks to only decoding data between the time rulers. This will allow us to focus only on CAN traffic during fuel pump operation

To summarize at this stage:

ID 3 35 disappears when the fuel pump controller is disconnected from the CAN network
ID 3 35 is transmitted on the CAN bus if the fuel pump is activated or static, however the payload is continually updated at 1 second intervals when the fuel pump is running.
ID 3 35 is first transmitted 50 ms before the in-rush current into the fuel pump (Trigger point)

ID 5 17 disappears when the fuel pump controller is disconnected from the CAN network
ID 5 17 is transmitted on the CAN bus (at 646 ms intervals) if the fuel pump is activated or static, however the payload never changes. (It remains fixed at 00 00 00 00 12 61 00 17)
ID 5 17 transmissions appear periodic in nature with no influence on transmission timing (646 ms intervals) or payload regardless of pump operation either via the scan tool, or with the engine running.

There is still not enough evidence to ignore ID 5 17 but at this stage we know ID 3 35 is our fuel pump controller, whilst remaining mindful of ID 5 17 as we continue decoding other ID’s.
FP Active test
(10.64 MiB) Downloaded 185 times
I will follow on ASAP with additional decoding techniques and features of Excel that will help group other ID’s relevant to their CAN Controller (Inc. ID 5 17)

Thanks for reading as I know its “heavy”….

I hope this helps, take care……Steve

Steve Smith
Pico Staff Member
Pico Staff Member
Posts: 740
Joined: Sun Aug 25, 2013 7:22 am

Re: Who's online and Who said what?

Post by Steve Smith » Mon Feb 25, 2019 11:12 am

The final part of this journey takes advantage of the unique electrical characteristics (electrical signature) of each CAN Controller and discusses the techniques associated with grouping CAN ID’s

The technique covers the last item on the list from the first forum post above
11. Use Excel which I will go through in the next post (As soon as I understand what I am doing)

N.B Ensure the vehicle under test has sufficient battery support, the CAN network physical layer to be functioning correctly (in terms of dominant and recessive voltage levels) and your CAN data is captured using the High Impedance Probes. topic21591.html?&p=97872#p97872

Before I go on, please read the following article from the team at Warwick Control Technologies as the details below replicate their reverse engineering techniques
https://www.warwickcontrol.com/press-re ... gineering/

Until I read the article above I had not considered the fact that each CAN Controller (ECU) would have an “electrical signature”, but I guess it makes perfect sense. As the article describes, when arbitration has been won by the relevant CAN Controller, it can then go ahead and transmit its message/payload on the BUS. At this point in time only one CAN Controller is transmitting as all others fall silent (they have lost priority)

The dominant voltages (CAN Hi 3.5 V & CAN Lo 1.5 V) present on the BUS during the transmission of the message/payload will be unique to the CAN Controller transmitting the message due to manufacturer tolerances within the ECU, connectivity, varying components and its physical location on the BUS

During the transmission of CAN Frames/Packets we can use PicoScope serial decoding to locate just the message/payload of each CAN frame for any ID and then use Deep Measure topic17711.html to obtain the unique dominant voltage values for CAN Hi and CAN Lo during the transmission of the message/payload

Once we have these values, we can export the Deep Measure data into Excel, obtain the modal value for the dominant CAN Hi & CAN Lo voltages and plot them in graph form to link known CAN ID’s with unknown ID’s. I must confess here, I did not know what a “Modal” value was until I got stuck into Excel. Basically it is the number that occurs the most in a sequence of numbers.
For example: CAN Hi dominant voltages during a message transmission might be 3. 555 V, 3.555 V, 3.661 V, 3.514 V & 3.601 V. The modal value would therefore be 3.555 V as it occurs twice in the sequence of voltages described here.

Starting with our known Shift Lever ID 1 97, we use our serial decoding “Filter” tool to locate ID 1 97 and the Time Rulers to partition the message/payload from the complete CAN Frame/Packet. We then apply Deep Measure to channels A and B “Between the Rulers” to obtain accurate values for our CAN message dominant voltages

Partition “message” from CAN Frame
IMAGE 1.png
Partition message
Apply Deep Measure “Between the rulers” for channel A and B. Below we have the procedure for the application of Deep Measure to channel B (CAN Lo) using 2 V as our crossing point or Cycle Threshold. Repeat this procedure for channel A (CAN Hi) using 3 V for our crossing point.
IMAGE 2.png
Apply Deep Measure
Below we have the dominant voltages (between the Time Rulers) captured via Deep Measure for CAN Hi during transmission of the payload for CAN ID 1 97. Repeat this procedure for CAN Lo looking at “Min Voltage” as these will be the dominant voltage values for CAN Lo
IMAGE 3.png
Dominant voltages
Using the “Export” button, save each Deep Measure .csv file to a location on your PC. To avoid confusion name the files 1 97 HI and 1 97 LO respectively. Open the .csv file and click on the empty cell below the relevant column of interest. For CAN Hi it will be the column header “Max Voltage V” and for CAN Lo it will be “Min Voltage V”
IMAGE  4.png
Locate Modal value
Click on “Recently Used” and select MODE.
N.B you may have to search for MODE using the “Insert Function” option, but once you have used MODE, it remains under the “Recently Used” button. This will then open the “Function Arguments” window which will automatically be populated with the cells for applying the MODE function. Click OK and the modal value will appear in the empty cell beneath Max Voltage V. For CAN Hi the unique modal value for our chosen ID 1 97 is 3.540776 V and for CAN Lo the value obtained was 1.491271 V.

Open a new Excel file and save these unique values to 2 new columns with the headings “CAN Hi & CAN Lo.” Repeat the above process for all the known and unknown ID’s you wish to qualify. In the second forum post above I could qualify ID 3 35 as fuel pump controller but there was a grey area surrounding ID 5 17.

With your new Excel file you can include additional columns with the headings “ID” and “Name” to act as a reference and provide a description for the known ID’s. Highlight all the unique values stored in the new Excel file, click on the Insert Tab and select a Scatter Chart. This will now plot all the highlighted unique values revealing the origin of unknown ID’s against known ID’s.

In the example below ID 3 35 (Fuel Pump Controller) has the same electrical signature as ID 5 17 suggesting that both ID’s originate from the same CAN controller.
IMAGE 5.png
Scatter Chart
In the article by Warwick Control Technologies they have used a clever technique of requesting a response from a number of ECU’s by sending standardized “diagnostic request” CAN ID’s/messages to various ECU’s (via a CAN data logger) and capturing the standardized response from the relevant ECU.
Think of the standard diagnostic request ID 7 E0 or 7 DF from a scan tool and the engine ECU response 7 E8.

If you then capture the modal unique dominant voltages of the response message (ID 7 E8) you can plot them in in your scatter chart along with other ID’s that form around this response. ID’s that cluster together around the same point within your Scatter Chart have the same electrical signature and hence originate from the same CAN controller

Given we don’t have this feature within PicoScope (the ability to generate diagnostic request ID’s /messages) we can use EOBD to request VIN data which will force a standardized response from the Engine ECU (7 E8) so providing us with “known” unique dominant voltages which we can then plot against other unknown ID’s
The following forum post will help here:

The video below describes the application of CAN decoding, Deep Measure, Export to .csv and the creation of a Scatter Chart within Excel as I know reading such detail raises questions that a video can answer.
https://drive.google.com/file/d/1L5vDdU ... sp=sharing

What was interesting about other ID’s I plotted in the Scatter Chart was the lack of ID’s around the engine response ID 7 E8 as I would have thought multiple ID’s would be clustered around this ID. Given the vehicle is a BMW F31, whilst the engine ECU is connected to the PT-CAN, communication for engine control is predominantly via FlexRay.

I am going to try this technique of plotting unique modal voltages with another vehicle utilizing a CAN network and so validate the process further.

Thank you again for reading, I hope this helps

Take care…….Steve

Post Reply