Anyone Tried Arduino for OBD or K-line sniffing or Broadcast

Ask for and share advice on using the PicoScope kit to fix vehicles here.

Anyone Tried Arduino for OBD or K-line sniffing or Broadcast

Postby FioranoCars » Mon Oct 17, 2016 11:49 am

Hi All

Following our recent, still ongong, battle with a certain Bull of a car, the ever lurking idea of getting deeping into CAN (and K line) sniffing/listening (useful for paradrain tracking, when the source of the drain is not the faulty unit - had this with a Bose Amp recently), and after looking at the available options, including some for £K's, I'm back to Arduino and either specific CAN shields or for K-Line a manual op-amp board or using single Motorola/Freescale chip (the MCZ33290EF).

So while I have a reasonable programming background (well over 40years, though not day to day or as a real job), and a level of understanding of CAN, a smattering of Arduino, and lots of TCPIP/SIP packet sniffing, I was hoping not to re-invent any wheels ...

There are many Arduino OBD projects out there, mostly just to display Engine Data as a mini dash, not really the sort of thing I'm thinking of doing. I'm looking to try and match Packet ID's to see who is talking, and log the data payload. There is much less K-Line stuff, and the 12v+ signal levels mean fewer people have played here, but we still see KLine as critical, while not as big as CAN, still very important.

Maybe in future being able to capture and replay diagnostic messages and their acknowledgements (PIS adjustments for clutches we don't have in house tools for, or service/airbag resets etc), but that's possibly a long way off (but certainly within the scope and part of the long terms goal)!!

Anyone had a toy with this, using Arduino or other boards, any Commercial products that I should consider before being a stone age idiot and make a lumpy wheel ... Any comments, ideas, sharing welcome.

Of Course any CAN based information, low level or at least covering low level to some extent, books, presentations, links etc (PM if you'd rather).

Based on interest, I might be prepared to publish the project on GIt-Hub or something, either privately or open.

Be good to hear any stories, boards that are problematic etc, absolutely anything!

thanks
Richard
User avatar
FioranoCars
Zen Master
Zen Master
 
Posts: 288
Joined: Thu Dec 13, 2012 11:26 am
Location: London, UK

Re: Anyone Tried Arduino for OBD or K-line sniffing or Broad

Postby steevegt » Tue Oct 18, 2016 1:00 am

Hi Richard

I have never tried a dedicated hardware solution for CAN or K-Line "debugging", but there are a number of manufactures out there that produce dedicated hardware for this type of situation.
Maybe the cheapest solution is, as you pointed, the CAN shields for Ardunino or Raspberry Pi.

If the CAN or K-line network that you are trying to decode, has the same signal protocol as the OBDII standard, I believe you may program some software for the ELM327 to do just that. Not really sure if the ELM327 allows for that, but I believe it may do the job.
You are stuck with the OBDII standard baud-rates and voltages, but it will work in some public and private networks out there, but not all I know...

The Pico, or other scope that has large memory and serial data decoding, is a good tool for this also. With the scope you have the disadvantage of a possible miss of data when changing buffers, and you can't write on the network.

This said, I will try to help, as far as I can and know, with some techniques that I have used before in trying to figure it out what is going on in the network.

First, trying to figure it out for who a random message in the network is destined for, as far as I know, is something near impossible. The message is broadcasted in the network, with some arbitrary number ID that the manufacture defined. Unless you have a documentation for that (not realistic), or you make sense of what is in the content of the message, you can't really know for who that is.
Sometimes you may have clues. For instance in CAN messages, the ID of the CAN frame, defines the message priority. The lowest the ID, the highest the priority. This can be a clue for finding something that is critical, for instance the data that must pass between the immobilizer and the ECU for the engine to start.

On the other end, knowing the source of a message, is something that is very easy.
The network wires are either a pull-up or pull-down design. When the source of the message tries to write something on the network, it will induce a current flow on the network wires, from, or to the module that is writing the message.

I had made the following capture on a CAN network, using a scan tool as an external can network node, and measuring the current on the CAN Low wire that went to the scan tool.
Channel A (Blue) CAN L voltage
Channel B (Red) CAN L current
can bus 1.PNG
CAN Bus Current


I have used a 10 mV/mA current probe. We are talking about 15 mA of current.
This types of low current probes are much slower that the Pico 20/60 Amp Clamp, and because of that we see one spike of current, when in fact we know that the current follows the same pattern of the signal voltage, but that is just too fast for that probe to capture.
But for the purpose of identifying the network talker, at 500 kbs/s, the speed of the probe is not a problem.
Using the scope for debugging in here, has the extra possibility of triggering on when a message is sent from a particular node.

An example as before, but for a LIN Bus network:
lin bus.PNG
LIN Bus Current


Hope to be useful
Thanks
Steeve
steevegt
User
User
 
Posts: 39
Joined: Tue May 12, 2015 10:05 pm

Re: Anyone Tried Arduino for OBD or K-line sniffing or Broad

Postby FioranoCars » Wed Oct 19, 2016 12:03 pm

Thanks Steve
Never considered the current consuption for ID'ing nodes on the CAN, I'll add that to the toolbox, thanks

So far, thanks to
Pico
Yes, sadly due to "gaps" between buffers, this is not suitable to listen and follow a "conversation", any data missed would be a failure, and render the analysis void. As a quick and dirty, is my CAN signal valid, it's great.

The ELM327 is according to "superior" sources too slow for this sort of task and other has limitations, not sure what, but that's the word, and I'll not double check others work, as hardware is relatively cheap anyway.

The Arduino (faster than most Pi's especially in terms of I/O operations) is also thought to be a bit slow if you get into complex stuff, and the recommendation for serious playing is the Intel® Galileo Gen 2 board, which is sort of compatible with the Arduino sketches, but has access to Visual Studio as well, so no doubt much more powerful, though still single core, single thread architecture. I'll probably hold off the Intel board as it's about 10x the cost, and I have a few spare Arduino floating.

Technical data/books/articles:
Thanks to all those who have contributed offline with links, downloads and files, please keep them coming, especially stuff about Physical Layer and Data Layer (voltages and timings) for K Line - but anything else on this side for different CAN specs welcome.

Also the Application Layer stuff (message structures, contents) and initialisation, although right now I'm planning on listening only, sooner or later it'll crop up as to why the initialistions failed, so understanding the wake up stuff is pretty important too.

Also anything (docs or sites) dealing with Arduino and OBD, there are quite a few, many with little recent activity, so if you stuble over something, please pop it my way! Ideally not projects to make a dashboard or fuel economy meter, as they are not really too relevant, but heck, I'm happy to filter!

Finally, any links to software for analysis of CAN/Data packets. I use Wireshark for TCPIP stuff, and could just write the data into a format that program can read, which I might do anyway to give the output an opensource universal viewer, but if there is anything (commercial or Free/Opensource) that is dedicated to OBD/CAN packets, then please let me know, this is probably the critical part of the jigsaw ... raw data is one thing, the analysis of THOUSANDS of message per minute is the value! Bear in mind some captures could be 10-60minutes long (LandRover vehicle shutdown is 45mins), road tests for intermittent faults ...

Will try and prototype something in next 4-5 weeks, let you know how it goes, but I want to let the theory soak in before I go blowing up any ECU's !

Thanks
Richard
User avatar
FioranoCars
Zen Master
Zen Master
 
Posts: 288
Joined: Thu Dec 13, 2012 11:26 am
Location: London, UK

Re: Anyone Tried Arduino for OBD or K-line sniffing or Broad

Postby steevegt » Thu Oct 20, 2016 12:27 am

FioranoCars wrote:Yes, sadly due to "gaps" between buffers, this is not suitable to listen and follow a "conversation", any data missed would be a failure, and render the analysis void. As a quick and dirty, is my CAN signal valid, it's great.


I don't think that is such a big limitation.
The "initialization" between computer modules in a car (if exists) shouldn't take more that a second or two, and should only happen right after you turn the ignition switch on or you open the car.
Any modern Pico Oscilloscope can record much more than that, with detail, in a single buffer.

The only limitation I see is when you want to capture a communication event, that it will happen only once, and that you don't know when it will happen or if it will happen.

I don't believe that you could have some type of communication protocol in cars, under CAN, K-line, ..., that you couldn't understand anything if you miss just one frame.
In a vehicle environment I don't believe that this makes sense.

FioranoCars wrote:The ELM327 is according to "superior" sources too slow for this sort of task and other has limitations, not sure what, but that's the word, and I'll not double check others work, as hardware is relatively cheap anyway.

The Arduino (faster than most Pi's especially in terms of I/O operations) is also thought to be a bit slow if you get into complex stuff, and the recommendation for serious playing is the Intel® Galileo Gen 2 board, which is sort of compatible with the Arduino sketches, but has access to Visual Studio as well, so no doubt much more powerful, though still single core, single thread architecture. I'll probably hold off the Intel board as it's about 10x the cost, and I have a few spare Arduino floating.


That may be true, I don't know for sure, never got my hands on that to prove otherwise, but I can't understand why they are slow for this.

Some car computer modules have a very low processing power, maybe lower than an Arduino board, and they do get the job done. They process network data, send network data and do there stuff in the middle.

You are talking about single core limitation and processing K-line data. K-line has a theoretical max speed of 10.4 kbps.
This is just a very stupid low speed value.
Even CAN networks at the most common 500 kbit/second, are a slow communication method, in modern computer power standards.

I will be very surprised if you find any limitation in recording all CAN data at 500 kbps with a regular Arduino board and a CAN shield.

FioranoCars wrote:Also the Application Layer stuff (message structures, contents) and initialisation, although right now I'm planning on listening only, sooner or later it'll crop up as to why the initialistions failed, so understanding the wake up stuff is pretty important too.


I can only say good luck with that.

I have before successfully decoded some non "scan tool data" in a car network, like the rpm value that goes to the instrument cluster, radio station name, button pressed, ...
All of this was possibility because I knew what I was looking for.
For instance, capturing the data and increasing the rpm at the same time, changing the radio station, or pressing a button multiple times...

What I have seen so far, regarding "Application Layer", is that there isn't (almost) any.
For instance, the ID that you have in the CAN message, represents the data "source" or content type. The meaning of the data in that frame is a very static thing.
As an example of something that I have "decoded" in CAN, in a Opel/Vauxhall Astra H, the RPM value is broadcasted in a CAN frame with ID 0110 (hex) or 272 decimal value.
The bits from 9 to 24 represent a 16 bit binary integer number of the RPM value.

In here you can see the decoded frames (just bits filtering), and a chart with the RPM that is "encoded" in the data.
can rpm.PNG
CAN RPM Decoded


I think it makes total sense for the manufactures to do things like this. If you add a more complex application layer, then you will need more computer power, for no practical reason.
More than that, you will increase the network overhead, in a "slow" type network.

FioranoCars wrote:Also anything (docs or sites) dealing with Arduino and OBD, there are quite a few, many with little recent activity, so if you stuble over something, please pop it my way! Ideally not projects to make a dashboard or fuel economy meter, as they are not really too relevant, but heck, I'm happy to filter!


Projects using something other than data that you can't get with the standard OBDII protocol I believe are really rare, and very limited in function.
Other than getting OBDII data, I have only seen little stuff, like using the steering wheel buttons as an input to an android tablet, and stuff like that.
I believe the reason is that OBDII is the only documented and standard communication protocol in cars.

More, I had read sometime ago, something about reverse engineering the data in car network to be illegal.
Don't really know for sure, but makes some sense to me, since because of this we could be producing documentation for an immobilizer bypass, odometer value change, MIF light always off, ...

FioranoCars wrote:Finally, any links to software for analysis of CAN/Data packets. I use Wireshark for TCPIP stuff, and could just write the data into a format that program can read, which I might do anyway to give the output an opensource universal viewer, but if there is anything (commercial or Free/Opensource) that is dedicated to OBD/CAN packets, then please let me know, this is probably the critical part of the jigsaw ... raw data is one thing, the analysis of THOUSANDS of message per minute is the value! Bear in mind some captures could be 10-60minutes long (LandRover vehicle shutdown is 45mins), road tests for intermittent faults ...


I believe Wireshark type software only makes sense if we are talking of a higher level protocol like TCP/IP, and you could get "metadata" from the overhead of the protocol.
In here, at least as far as I have seen, the only stuff that you have is the network related data (ID, Checksum, CRC, ...) and the raw data bytes.

FioranoCars wrote:Will try and prototype something in next 4-5 weeks, let you know how it goes, but I want to let the theory soak in before I go blowing up any ECU's !


Please keep us informed with your progress, and I'm very curious to see your approach to this.
steevegt
User
User
 
Posts: 39
Joined: Tue May 12, 2015 10:05 pm

Re: Anyone Tried Arduino for OBD or K-line sniffing or Broad

Postby Mark Dalton » Tue Oct 25, 2016 3:56 am

If your at all familiar with Linux or have some one in know that may be able to help you, the method spoken about in this you tube video looks pretty handy and the interface they used isn't too expensive. https://www.youtube.com/watch?v=oHqDLWN3a_w
They talk about using different interfaces in the video, including ELM327. I've had a little play with this and ELM327 interfaces were, for me anyway, impossible to get connected to the Linux system. I tried with a couple of other interfaces I've got and they didn't work either, but that could be my total lack of expertise in this area.
But this is an area of great interest to me as well, so would be great to hear how you go.
User avatar
Mark Dalton
Advanced User
Advanced User
 
Posts: 60
Joined: Mon Dec 01, 2014 9:35 pm
Location: Melbourne

Re: Anyone Tried Arduino for OBD or K-line sniffing or Broad

Postby FioranoCars » Tue Oct 25, 2016 9:50 am

Hi Mark
Linux I'm ok, probably rather not if I can help it as all our workshop systems are Windows, although Python appears to be a weapon of choice for many hackers, which is cross platform, so cool with that, just a case of hardware drivers, and sure that will be ok, so it'll be portable for those who want to take back to Linux :D

The OpenGarages boys are interesting, definately a site I need to crawl through and learn from.

Sadly in Onion mode still, research has shown lots of options with each having different restrictions, and a few other people posting about their own attempts at such projects. So about to start a document to list the sources etc, I'll probably share or do a blog article about it, although "our" purpose is neither hacking the car (security features or actual control of systems like brakes/steering) or the basic stuff like speed and fuel with GPS which are 2 very common "clusters" of project types, or even the other main cluster of simple DTC read and clear.

It looks like there is very little need to understand the lowest levels of the CAN, and plenty of chipsets will do it all for you, application framework is likely as low as you need to understand, bar maybe dealing with odd systems, when using a tool like Pico will show init packets and speeds, to allow custom fixes. That will help greatly.

I guess I'm going to bottom out the research phase before I do a great deal, but will be doing some initial tests with Arduino and multiple MCP2515 boards, each for single CAN (pair) to get a feel for data volumes and possibly some basic command "replay" to see how easy it is to sniff, decode, understand and replay basic Diagnostic tool commands. We still have a reliance on Kline 12v and the MCP2515 will not tolerate those voltages. (historic needs, as we still see lots of late 80's and early 90's cars which are as important to us modern stuff).

As I want to do far more than this, I'm going to crawl first and see what benefits can be achieved and are within easy reach, before prototyping more complex solutions/coding :D

I'm almost certain that it will be 2 bits of kit, as Key to my plans are:
- multiple CAN stiffing (entire OBD port 13 pins) plus maybe 1 or 2 extra pairs on top (2-4 more wires)
- K line 0/12v and CAN 0/5v + 2.5v configs ... plus future stuff (optical as well)
- External trigger/marker insertion (to log event by operator) or voltage/current, maybe even can message if I get good!

So
1. a broad multi channel sniffer (ie take a car to the main dealer and record all activity EOBD (and some private CAN's on top) covertly with the kit small enough to remain unnoticied). Lets call this "the Sponge"! Definately not PC based. Probably more than 1 external trigger, capable of logging for extended timeframes. Possible remote GSM/Bluetooth interface. Might even build in some filtering capability (packet frames, not noise!).

2. a more precise bit of kit, with probably 2 can pairs and 2 k-lines, maybe multiplex-able pin outs (maybe an existing programmable interface) that can read and write, etc.

Quite a few things still to understand, hardware wise inc buffering/local SD card storage, hence going to break myself in gently, then refine what I want to do, and avoid wheel making where I can, provided there is no mission critical compromise.

I do have an electronics guru friend for fancy component engineering (rocket scientist, yes!) or simple capacitor calculationsbut but will avoid as much of this as I can, because it's clear there is a huge amount of stuff already out there, although the Chiop i wanted for K-line stuff has gone End of life, and I can't find something similar right now, just need to research a bit I guess.

If you think I'm missing something shout, or indeed bitting off Moby when I should be settling for a sardine, I'm all ears!

Richard
User avatar
FioranoCars
Zen Master
Zen Master
 
Posts: 288
Joined: Thu Dec 13, 2012 11:26 am
Location: London, UK

Re: Anyone Tried Arduino for OBD or K-line sniffing or Broad

Postby Mark Dalton » Tue Oct 25, 2016 10:13 pm

What your shooting for sounds like what a lot of Techs. out there would be interested in. Like you say there seems to be two ends of the scale. One very basic, that's not much use diagnostically and the other end that far more complex than we need since it's as much about defeating the safety protocols to take over control of the vehicle, than understanding faults.
It's a topic that gets me all excited, then start doing a bit of research, get overwhelmed at the scale of the task and then fall in a heap for a while. I live in GM dealer land, so on top of the other issues, GM uses GMLAN which is a CAN/ISO-TP mish mash. They don't make it easy!
I look forward to hearing about your adventures........
User avatar
Mark Dalton
Advanced User
Advanced User
 
Posts: 60
Joined: Mon Dec 01, 2014 9:35 pm
Location: Melbourne


Return to Diagnostic discussion

Who is online

Users browsing this forum: No registered users and 1 guest