2013-07-29

generating RTL2832U driver code

Sniffing and analysing communication protocol

I decided to put online my RTL2832U reverse-engineering / hacking scripts. I get quite often questions how to sniff USB, analyse data and generate some driver code to test. I hope this example helps. Fortunately RTL2832U USB protocol is also very simple, which is easy to read and understand even with very limited existing knowledge.

I like to use native Windows XP and SniffUSB 2.0 to took USB captures. There is multiple USB sniffers available, both software and hardware. Feel free to select one you like, but be aware that these scripts only work for SniffUSB logging format. Making log format conversion scripts should be quite trivial though.

Step-by-step

  1. Take sniff using SniffUSB (outputs file UsbSnoop.log). Just tune to working TV channel, one sec is enough, log file size increases very rapidly when picture is streaming.
  2. Use parser.pl to convert SniffUSB log (UsbSnoop.log) more human readable form. (History: In my understanding parser.pl was a part of USBreplay toolkit.)
    • ./parser.pl UsbSnoop.log > UsbSnoop.ts
  3. Lets remove video data as we are not interested of it. It is only control data what we like to get.
    • sed -e 's/BULK\[00081\].*$/BULK\[00081\] MPEG2 TS packet data removed/g' UsbSnoop.ts > UsbSnoop.p
  4. Dump out register writes from the control data.
    • ./rtl2832u.py UsbSnoop.p > UsbSnoop.c

rtl2832u.py could currently extract all registers from rtl2832u devices that has tuner e4000, fc2580, fc0013, tua9001 or r820t. There is few more RF tuners used with rtl2832u and in case of unknown tuner it could still dump out rtl2832u itself registers. Generated code could be copy & pasted directly to the Linux drivers in question and make some tests.

Bug hunting

I have found that approach most easiest way to find out and kill Linux driver bugs. Lets take for example RF tuner bug, which causes notable reduced sensitivity. Took sniffs from the working machine, generate code, copy & paste that to the suitable places of non-working driver. Rapidly that non-working driver starts working and it is time to do manual binary search to find out problematic setting and fix it. Easy? Eventually yes.

Scripts and an example (rtl2832u + fc0013)

http://palosaari.fi/linux/v4l-dvb/rtl2832u_scripts/

9 comments:

  1. Another trick I commonly use for this sort of thing is to take the trace, and then have a Perl script find the last value of each register set for a given chip. This allows you to build up a representation of the internal state of the chip under Windows and compare it against a register dump under Linux.

    Devin

    ReplyDelete
    Replies
    1. Yeah, building shadow registers by your script. That is very useful in case of you are making 100% reverse-engineered driver from the bus sniffs.
      There is quite a lot of small tricks to find out different data.
      For example demod has only few registers which are programmed differently for each bandwidth. Those are easy to find out taking sniffs against DTV signal generator and then diff programmed registers.
      TPS (transmission signal parameters) could be found just dumping all demod registers and check which are changing according to parameters set to signal generator. But for these you will need that modulator...

      Delete
  2. I prefer to use VirtualBox USB pass-thru and get the trace directly from the Linux USB debugging interface. One advantage of this is that the log comes out directly into the Linux system without having to transfer it around. Another advantage is that I can knock together a fake emulation of the hardware using gadgetfs and the loopback driver, and then use it to inject data into the Windows driver and see how it responds.

    What advantages does SniffUSB have?

    ReplyDelete
    Replies
    1. Advantages of SniffUSB are ... not very rational. It is simple, free, does the job and is popular. Also I have used it from the very beginning of driver hacking career ~2006. Same applies for most LinuxTV developers I think. But disadvantages are coming larger everyday as it supports only legacy Windows XP. I actually has a Ellisys hardware sniffer too, but I am too lazy to use it because of SniffUSB just works :)
      I have also tried virtual machine solutions, both VMware and VirtualBox.VMware tends to be broken after each Kernel upgrade as its Kernel driver patch touches Kernel sources deeply. Biggest problem with VirtualBox was missing USB2.0 support! Otherwise VirtualBox could be very nice solution.
      Dunno if there virtual machine things has changed during last ~3 years...

      Delete
    2. In my experience the use of virtual machines for USB debug has been a mixed bag. It works reasonably well nowadays for digital TV (where the bandwidth is typically only 10-30Mbps), but still falls down when trying to debug raw analog TV (where the data rates are on the order of 240 Mbps).

      Don't get me wrong - it *has* gotten better. When I first tried debugging a tuner with VirtualBox back in 2009, it locked up the entire system.

      Devin

      Delete
    3. I see, thanks. I've only really hacked on quirky HID devices and other low-speed, USB 1 stuff, so never run into these issues.

      Delete
  3. Hi Antti,
    With your trick , i found a problem in your E4000 module, there is a problem in the sigma delta calculation.
    i've fixed the problem with the folowing code

    /*Problem in sigma delta
    sigma_delta = 0x10000UL * (f_VCO % priv->cfg->clock) / priv->cfg->clock;
    */
    //Fixed
    f_VCO = c->frequency * e4000_pll_lut[i].mul;
    xl=priv->cfg->clock;
    sigma_delta = (f_VCO % xl);
    sigma_delta=sigma_delta/1000;
    xl=xl/1000;
    sigma_delta=sigma_delta*65536/xl;
    //end of fix

    With your code the sigma delta has always a small value due to a bad convertion.
    The problem was an improper tuning on some frequencies

    I've send you a email with my full code
    Best Regard's
    Damien CABROL

    ReplyDelete
    Replies
    1. Nice! I will check that on end of the week as I am on holiday trip currently.

      Delete
    2. Problem was 32-bit overflow in calculation. The reason why I have not been able to find that, even there has been very many bug reports, is that I am using 64-bit Kernel and bug appears only in 32-bit. I am now waiting patch from Damien...

      sigma_delta = 0x10000UL * (f_vco % priv->cfg->clock) / priv->cfg->clock;
      sigma_delta = div_u64(0x10000ULL * (f_vco % priv->cfg->clock), priv->cfg->clock);

      Just that bug was the main motivation behind that blog post (as I mentioned in one mail in the mailing list) and seems like it leads the bug fix!

      That makes also wonder what I should do in order to improve situation. The fact is that almost every driver I have made has had similar 32-bit vs. 64-bit bugs. Lastly it was Mirics MSI3101 driver one week ago, which Mauro fixed kindly, as I was travelling at the time :) I don't want to invest any new hardware only for testing all kind of architectures...

      Delete