Let me begin with a snapshot of CAN traffic:
Here we have captured and will be able to evaluate the integrity of the CAN signals (the physical layer). The speed and sample rate of the scope allow us to look at the CAN signals as though we were looking through a microscope.
This capture shows us a glimpse of two CAN packets travelling through a twisted pair of CAN bus wires, revealing the correct upper and lower voltage levels (3.5 V— CAN Hi and 1.5 V — CAN Lo) along with the correct “idle” voltage of 2.5 V.
Note how the voltage “ringing” and “over-shoot” below is typical of automotive CAN signals and can be attributed to a number of factors such as the style of test lead (standard or high impedance) environmental noise and “end of message” confirmation.
Remember that none of these are faults and CAN networks are immensely fault tolerant when it comes to the physical layer. Here is a typical example: https://www.picoauto.com/support/topic20981.html?&p=93851&hilit=fault+tolerant#p93851
With the capture above being similar to seeing the flowing CAN traffic through a microscope, the capture below can be compared to looking at flowing CAN traffic from a distance, but we still have the option to view it in greater detail if necessary, thanks to a pair of binoculars (the zoom feature).
To put this volume of CAN data into perspective, the time across the screen is just one second (100 Ms/div x 10 div). By using the CAN decode feature in PicoScope we conclude a total of 1066 messages in just one second of passing traffic.
To display the decoded CAN data, follow the steps in the image below:
If you are unsure about the Baud rate (network speed), you can try the technique in the following image:
Note how the decoded example above presents a warning to the user that the “sample rate may be too low” (the Properties window indicates 1 MS/s when the ideal would be a minimum of 2 MS/s), which consequently may lead to erroneous CAN frames highlighted in red within the decode table.
To avoid this scenario, we can use a number of software and hardware features to ensure a superior sample rate (sufficient to decode high-speed CAN).
There are a number of decoding techniques listed in these resources:
Focusing further on CAN decoding, how can we ensure a sufficient sample rate to provide efficient serial decoding of high-speed CAN, while also allowing a practical time span on the screen to improve CAN controller identification? Remember the “end game” here is to reveal the IDs associated with a number of CAN controllers.
The first tip is to increase the time span across the screen and the number of selected samples. Below I have used 500 MS/div (the total time across the screen is 5 seconds) with 10 MS which delivers 2 MS/s to ensure the integrity of the decoded CAN data.
10,000,000 samples across the screen / 5 seconds on screen = 2,000,000 S/s
Next, rather than to take up two channels with CAN data, if we are happy with the formation of the CAN signal (see image 1), we can use Channel A to capture the CAN Hi signal via a High Impedance probe (TA375) to minimise noise, ringing and signal intrusion (protect the integrity of the CAN signal).
You can find more information about the High Impedance Probe here: topic21591.html?&p=97872#p97872
The capture below highlights how the software settings and chosen hardware has a positive effect on the capture of the CAN Hi network signal. While the software will still warn users of a possible sample rate concern, you can see that there are no error messages in the decode table. Also, note how we now have a total of 5318 CAN frames to wade through.
The capture above is from a 2013 BMW 2.0 Diesel F31 Auto, where the shift lever indicates the gear position via the high-speed CAN network. (The gear lever is, in effect, a CAN controller)
Channel B had a current clamp (TA234 https://www.picoauto.com/products/curre ... rent-clamp) attached around the shift lever CAN Hi branch wire to act as a “location finder” for any transmissions sent from the shift lever.
To clarify, where we have a voltage differential (CAN Hi – CAN Lo = 0 V – 2 V) there will be current flow (the CAN bus load is approximately 60 ohms) from the shift lever when messages are transmitted onto the CAN bus network. Note the sequential spikes from the current clamp.
By using the zoom, time rulers and scaling features, we can measure the time span between each spike captured by the current clamp (approximately 30 ms). If you position the mouse pointer over the decode bar (at either time ruler/current peak) in the PicoScope display (graph), a pop-up window will show information about the CAN packets in the area the mouse pointer is pointing to.
This technique depends on how steady you are with the mouse pointer, so I would recommend zooming directly above one of the time rulers and increasing the scale of Channel B (x 50) to align the current spikes on Channel B with each CAN packet on Channel A. Notice how the peak current flow (from the shift lever’s CAN Hi branch wire) always coincides with a transmission from a CAN controller with the ID 1 97, and that these transmissions occur every 30 mms. It is starting to look like the shift lever has the CAN ID 1 97.
Can we use this knowledge to categorically determine, not only the CAN IDs of the shift lever, but also other CAN controllers using the same technique?
Before we move forward with this, there are a few pointers to be aware of with CAN IDs:
The CAN ID serves a number of purposes: It informs of 11 or 29 bits CAN and the priority of the pay load (data). The lower the numerical ID the higher the message priority. The same CAN controller can also produce multiple IDs.
For example: Engine coolant temperature ID may be grouped in “Temperature Related” messages, and the engine speed ID could be grouped with “Engine related” messages. Both IDs have come from the same CAN controller.
A clue to the identification (ID) of a CAN controller can be found in the “Arbitration” given arbitration is won based on message priority. For example, if we discover the byte within the data field that denotes the “Brakes ON or OFF” message, we could assume this data most probably originated from the ABS CAN controller, and so we have now identified the ABS CAN controller ID. This is not, however, an exact science as the “Brakes ON or OFF” message could also come from the Cruise Control CAN controller.
Basically, you should not be surprised if you discover multiple IDs that originate from a single CAN controller (ECU).
I mentioned earlier how current flow from the shift lever occurs every 30 ms. Let us correlate these transmissions (current flow) within the CAN decode table using the “Filter function”.
In the decode table, click on the Filter button and enter 197 above the ID column to list only the CAN packets relevant to ID 1 97.
Note how the Start and End time of each transmission from CAN controller ID 1 97 is identical to the spikes detected by the current clamp (30 ms intervals).
It is also essential to filter the current clamp signal, as we are pushing the limits of the 30 A current clamp when measuring approximately 10 mA.
Looking at the data field in the decode table (highlighted above in yellow), we can see how the first 3 bytes of data are constantly changing with no activation of the shift lever (I am not sure why).
If we were to operate the shift lever now, we can look for a change in the data field (filtered column 197) that would help confirm that we have correctly identified the shift lever as CAN ID 1 97.
Now, PicoScope is not a CAN decoder in the true sense of the word. We cannot collect continual streams of endless CAN data like you can with devices such as the Intrepid Vehicle Spy or X-Analyzer 3 loggers and software suites.
PicoScope decodes the physical layer, while the CAN loggers decode the silicone layer within the CAN controller. What we have on screen and in the waveform buffer, is all we have to work with, and so we have to get clever when trying to identify CAN IDs.
By adding Channel C, I placed a pressure switch on top of the shift lever “Park” switch to indicate when the Park switch was activated in our displayed waveform. I used an NVH interface (TA259) to provide power to a micro switch, which in turn provided an output to PicoScope via the TA259 BNC output. Use this technique with the ignition on but engine off to minimize CAN activity and, of course, make sure that a battery support is installed.
From the decode table, select “Display Packets Between Rulers” and “Filter” for ID 197. Scroll down the decode table and look for a change in the data bytes relative to the operation of the Park switch.
Opening the Park switch forces a byte change from the shift lever on the CAN bus where we can see the fourth data byte change from C0 to D5 (press Park) and return to C0 (release Park).
As the shift lever transmits data every 30 ms, there is no change in the frequency of the current spikes. I was expecting there to be an instant current spike when pressing the Park button. I guess 30 ms transmission intervals is sufficient to ensure that any change in the shift lever status/operation is executed in a timely fashion.
All in all, there is a lot to digest about CAN IDs and there are other techniques, one of which is CAN controller/ECU disconnection: topic21593.html?&p=97888#p97888
An understanding of the CAN network and the structure you are diagnosing is essential and an accurate CAN topology or wiring diagram from the vehicle manufacturer is essential. These will reveal CAN “branches” and ensure the CAN line in which you are measuring current is only capturing current from one source.
Remember that PicoScope is far more than an oscilloscope. Among its numerous applications is a serial decode function. While it is not a dedicated serial decoder, we can manipulate the software to deliver decode/logger features. Knowing its strengths and weaknesses and applying the techniques mentioned in this article will help.
Let’s summarise the techniques mentioned so far:
More information regarding point 9 and 10:
Before you continue, make sure that the ignition is on, engine is off, and a battery support is installed. Below we have the low pressure, fuel priming pump control unit disconnected from the CAN network. After the scan tool’s ECU search, you can see that Fuel control is missing from the list.
Now, we use PicoScope to capture the CAN network.
Next, we connect the fuel pump again and run the same ECU search and grab the same capture with PicoScope.
Decode the data for all buffers in both captures (see image 4) and export this data to a location on your PC. This will save two .csv files to your set location.
Open the exported .csv file that contains the data where all the ECUs are online. Click on column E to highlight the entire list of IDs. Right click and select Copy. Click on column Q, and paste your copied content there. Now column Q will contain all IDs captured while all ECUs were online. To avoid confusion, it is good practice to give column Q a title. I have renamed it “ID FP ON”.
To reduce the amount of data in column Q, click on the Data tab and select Remove duplicates, (this removed over 10,000 IDs) and click Sort A-Z to sort the entries in a logical order.
Next, open the exported .csv from the disconnected fuel pump, right click on Column E and copy the data to column R in the other spreadsheet, next to the column with online ECUs. Repeat the process to remove duplicates and sort the entries in column R as well.
Finally, click on the Home tab, highlight columns Q and R and select Conditional formatting, Highlight Cells Rules, Duplicate values, and select Unique values. This will highlight a number of IDs that were present when the fuel pump controller was online, but were missing when it was offline. Copy and paste these highlighted IDs into column S. These are our IDs of interest.
We have identified the following IDs:
|2 26||2 F8||3 35|
|3 67||4 4C||5 17|
These were all missing when the fuel pump controller was disconnected. The ID 0F FF BF B9 appeared when the fuel pump controller was disconnected, I am currently not sure why, but will dig deeper.
The next step is forcing the fuel pump to activate with a scan tool while capturing the fuel pump current and CAN Hi data, simultaneously in a single waveform.
Use the Decode filter option and enter each ID above the ID column to determine the relevance of the ID during fuel pump operation. Starting at the top, 2 26 and 2 F8 could not be found. 3 35, however, appeared four times in five seconds, at approximately one second intervals (look at Start time). Interestingly enough, the first transmission from ID 3 35 was 50 ms before the in-rush current to the fuel pump. Also note how the Data column indicates a change in the information carried in the payload, suggesting an information update is taking place.
Moving on, ID 5 17 was the only ID that was present during fuel pump operation, with transmissions every 646 ms. Note how there are no changes in the data field during fuel pump operation. This suggests that the data is not relevant to the operation. However, this is an assumption, and it can therefore, not be discounted 100%.
We are slowly narrowing down all the IDs to just two possibilities — 3 35 and 5 17. By using the Decode between rulers option, we can begin to pinpoint the transmissions of CAN IDs that are relative to fuel pump activation, giving us the opportunity to judge the relevance of these IDs.
The image above shows how we reduce the CAN data even further by only looking at data between the time rulers. This allows us to focus only on CAN traffic during fuel pump operation.
To summarise at this stage:
There is still not enough evidence to ignore ID 5 17, but at this stage we know that ID 3 35 is our fuel pump controller. We will keep ID 5 17 in mind as we continue decoding other IDs.
FP Active test: TEST 9 ACTIVE TEST FB LOW FLOW.psdata
Next, we’ll look at additional techniques and features of Excel that will help group other IDs relevant to their CAN controller, including ID 5 17.
We take advantage of the unique electrical characteristics (electrical signature) of each CAN controller and will look at the techniques associated with grouping of CAN IDs.
NB! Make sure that the vehicle you are testing has sufficient battery support, that the CAN physical layer is functioning correctly in terms of dominant and recessive voltage levels, and that your CAN data is captured with the High Impedance Probes (topic21591.html?&p=97872#p97872).
I recommend you read the following article from the team at Warwick Control Technologies, as the details below replicate their reverse engineering techniques: https://www.warwickcontrol.com/press-re ... gineering/
Until I read the article, I had not considered the fact that each CAN controller (ECU) would have an “electrical signature”, but it makes sense. As the article describes: when arbitration has been won by the relevant CAN controller, it can go ahead and transmit its message/payload on the BUS. At this point in time only one CAN controller is transmitting as all others fall silent (they have lost priority).
The dominant voltages (CAN Hi 3.5 V and CAN Lo 1.5 V) that are present on the BUS during the transmission of the message/payload, will be unique to the CAN controller transmitting the message due to manufacturer tolerances in the ECU, connectivity, varying components and its physical location on the BUS.
During the transmission of CAN frames/packets, we can use PicoScope’s serial decoding to locate just the message/payload of each CAN frame for any ID and then use Deep Measure (topic17711.html) to obtain the unique dominant voltage values for CAN Hi And CAN Lo during the transmission of the message/payload.
Once we have obtained these values, we can export the deep measure data into Excel, find the modal value for the dominant CAN Hi and CAN Lo voltages, and plot them in graph form to link known CAN IDs with unknown CAN IDs. I must confess that I did not know what a modal value was until I got stuck into Excel. Basically, it is the number that occurs the most in a sequence of numbers. For example: CAN Hi dominant voltages during the message transmission might be 3.555 V, 3.555 V, 3.661 V 3.514 V and 3.601 V. The modal value would, therefore, be 3.555 V as it occurs twice compared to the other numbers that only occur once each.
Starting with our known Shift lever ID 1 97, we use the serial decoding filter tool to locate ID 1 97 and the time rulers to partition the message/payload from the complete CAN frame/packet. Then we apply Deep measure to Channels A and B between the rulers to obtain accurate values for our CAN message dominant voltages.
Partition message from CAN frame:
The image below illustrates the procedure for applying Deep measure to Channel B (CAN Lo), using 2 V as our crossing point or Cycle threshold. Repeat this procedure for Channel A (CAN Hi) with 3 V as the crossing point.
The following image shows dominant voltages between the time rulers, captures via deep measure for CAN Hi during transmission of the payload for CAN ID 1 97. Repeat this procedure for CAN Lo looking at minimum voltage, as these will be the dominant voltage values for CAN Lo.
Now we use the Export button to save each Deep measure .csv file to a location on your PC. To avoid confusion, I recommend you name the files 1 97 Hi and 1 97 Lo respectively. Open the .csv files and click on the empty cell below the relevant column of interest. For CAN Hi it will be the column header “Max voltage V” and for CAN Lo it will be “Min Voltage V”.
Click on “Recently used” and select MODE.
Note: You may have to search for MODE by using the Insert function option, but when you have used it once it remains under the Recently used button. This will open the Function arguments window which is automatically populates the cells for applying the MODE function. Click OK, and the modal value will appear in the empty cell you chose. For CAN Hi, the unique modal value for our chosen ID 1 97 is 3.540776 V and for CAN Lo the value was 1.49171 V.
Open a new Excel file and save these unique values to two new columns with the headings “CAN Hi” and “CAN Lo”. Repeat the process for all the known and unknown IDs you want to qualify.
Previously I explained how I could qualify the ID 3 35 as fuel pump controller, but that I was more uncertain about ID 5 17. In the new Excel file, you can include additional columns with the headings “ID” and “Name” to act as reference and provide a description for the known IDs. Highlight all the unique values stored int eh new excel file, click on the insert tab and select a Scatter chart. This will plot all the highlighted unique values and reveal the origin of unknown IDs against known IDs.
In the following example, ID 3 35 (the fuel pump controller) has the same electrical signature as ID 5 17, suggesting that both IDs originate from the same controller.
The article by Warwick Control Technologies explains a clever technique of requesting a response from a number of ECUs by sending standardised “diagnostic request” CAN IDs/messages to various ECUs (via CAN data logger) and capturing the standardised response from the relevant ECU. Think of the standard diagnostic request ID 7 E0 or 7 DF from a scan tool and the engine ECU response 7 E8.
If you then capture the modal unique dominant voltages of the response message (ID 7 E8) you can plot them in your scatter chart along with other IDs that form around this response. IDs that cluster together around the same point in your scatter chart have the same electrical signature, and can therefore be attributed the same CAN controller.
As we do not have the ability to generate diagnostic request IDs/messages in PicoScope, we can use EOBD to request VIN data which will force a standardised response from the engine ECU (7 E8), providing us with “known” unique dominant voltages which we can then plot against other unknown IDs. The following forum post will help with this: https://www.picoauto.com/support/topic21593.html?&p=97888#p97888
This video describes the application of CAN decoding, Deep measure, export to .csv and the creation of a scatter chart in Excel.
An interesting thing I noted about the IDs I plotted in the scatter chart was the lack of IDs around the engine response ID 7 E8. I would have thought that multiple IDs would be clustered around this, given that the vehicle is a BMW F31 while the engine ECU is connected to the PT-CAN, communication for engine control is predominantly via FlexRay. I will be trying this technique of plotting unique modal voltages on a vehicle that is utilizing a CAN network so that I can validate the process further. I will get bask to this at a later time.