Mirics MSi3101 SDR Linux driver

Mirics MSi3101

MSi3101 is name of the reference design that consists of two parts, USB ADC and RF-tuner. Used USB ADC is MSi2500 and RF-tuner is MSi001. That designs was introduced somewhere in 2009 aimed for PCTV marked. It is the first and the only one SDR based PCTV solution, which unfortunately never becomes very popular.

Mirics MSi3101 devices

That solution differs notably from the other PCTV solutions as there is no hardware demodulators, instead signal processing is done by host computer software demodulator. That reduces complexity of hardware but downside is increased complexity of software and needed CPU power.

If we compare traditional DTV USB solution and that software solution, we could see that kind of data paths in higher level:

HW, traditional, solution:
antenna => RF-tuner => HW demodulator => USB-interface => PC => TV-application

SW, SDR, solution:
antenna => RF-tuner => ADC => USB-interface => PC => SW demodulator => TV-application

The very first DTV USB ADC

First of all, Mirics MSi2500 USB ADC is very interesting chip! It is very first plain USB ADC I have ever seen, designed for USB2.0 SDR based digital television. Requirements coming from the DTV terrestrial reception means that there must be sampling rates around 10Msps and sampling resolution 10-bit or more. These are hard requirements to meet as all the bits are transferred over rather slow speed USB2.0, which becomes quickly to bottleneck. To tackle that problem chip offers multiple stream format - for more you need sampling rate the less you has resolution and the other way round.

reverse-engineering USB ADC sampling rate formula

Unfortunately I was not able to get any needed register documentation for that chip, which forces me to jump reverse-engineering mode. Whilst I have done reverse-engineering quite a hell much, I don't like to do it as it is simply a waste of time... Fortunately ADCs are rather simply chips as there is not much moving parts, in a good luck only sampling rate. In that case there were mostly sampling rate and stream format which are needed to find out in order to operate chip somehow meaningful.

I thought some time how I could find out ADC sampling rate calculation formula and eventually ended up developing following method. It is mainly based of register value brute-force with sample rate calculation, but contains mixture of some other steps. I could imagine that same method could be useful later, when new USB ADC chips are arriving to market and documentation how to configure ADC sampling rate is not known.

1. Prerequisites. You must have somehow working driver skeleton which streams ADC data. RF tuner functionality is not needed, but without the tuner step 2 could be hard to pass.
2. Learn stream format. Implement some logic to driver which calculates all I/Q samples in given period. After the each period calculate sample rate (by driver) and output it to the debug log. I used 5 second period in my tests - more longer period you use, the more accurate sample rate calculation is.
3. Find out registers involved to sampling rate setting. Sniffing and diffing... For example tune TV to 8 MHz channel, then 6 MHz and test even radio, depending on what app supports. Should not be very hard.
4. Add "write_reg" V4L2 private control to your driver and some appropriate implementation for it. You could guess why and how - if not that method is not for you.
5. Start streaming, eg.
   cat /dev/video0 > /dev/null
6. Brute force different values to sampling rate registers and see what driver reports as calculated sample rate.
   v4l2-ctl -d /dev/video0 --set-ctrl=write_reg=0x1234503

Fractional-N frequency synthesizer

Basic knowledge of Fractional-N based frequency synthesizer functionality is needed in order to succeed with that method,  a lot of patience and a little bit luck too. It is easiest to try found meaning for something that changes a lot output sampling rate - in practise it is output divider (Rout, LO div). After that, integer part "N" of PLL multiplier and finally fractional part, which is most hard.

Sample rates measured

Both devices seems to perform very similarly on sampling rate test made. Sampling rate limits were between 1.21 Msps and 15 Msps. Smallest stable sampling rate is a little bit high what I expected, but not too high.

Sampling resolution & stream format

I found 4 different stream formats, all having different resolution. As a USB2.0 device there is direct relation between stream format and sampling resolution and indirect relation for sampling rate. Stream formats known currently supports 14-bit, 12-bit, 10+2-bit and 8-bit resolutions - at least sample bit count got is that, effective resolution (enob) could be a bit less...

I calculated following ~theoretical maximum sampling rates per stream format using assumption maximum USB2.0 effective throughput is 280 Mbit/s.

format	resolution	sample rate
252	14	8 613 281
336	12	11 484 375
384	10+2	13 125 000
504	8	17 226 562

These results are very theoretical and not tested.

Btw. that stream format register didn't liked injection of new register values when streaming was ongoing :) Result was typically broken stream and it didn't get fixed until device was physically replugged. There could be still some more formats available and I suspect using own firmware you could implement your own formats. At least these are supported by default firmware.

There was also bits to select USB URB Isochronous micro frame size from one of the 1024/2048/3072. 3072 is maximum what USB2.0 supports and I don't see any idea to use smaller as bus datarate is so high in any case.

MSi2500 advertises on USB interface device descriptor that it has some more alternative settings to offer. That same register is likely place to configure those too.

USB device descriptor

Bus 002 Device 048: ID 1df7:2500 Device Descriptor: bLength 18 bDescriptorType 1 bcdUSB 2.00 bDeviceClass 255 Vendor Specific Class bDeviceSubClass 255 Vendor Specific Subclass bDeviceProtocol 255 Vendor Specific Protocol bMaxPacketSize0 64 idVendor 0x1df7 idProduct 0x2500 bcdDevice 2.00 iManufacturer 0 iProduct 0 iSerial 0 bNumConfigurations 1 Configuration Descriptor: bLength 9 bDescriptorType 2 wTotalLength 66 bNumInterfaces 1 bConfigurationValue 1 iConfiguration 0 bmAttributes 0x80 (Bus Powered) MaxPower 400mA Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 0 bAlternateSetting 0 bNumEndpoints 0 bInterfaceClass 255 Vendor Specific Class bInterfaceSubClass 255 Vendor Specific Subclass bInterfaceProtocol 255 Vendor Specific Protocol iInterface 0 Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 0 bAlternateSetting 1 bNumEndpoints 1 bInterfaceClass 255 Vendor Specific Class bInterfaceSubClass 255 Vendor Specific Subclass bInterfaceProtocol 255 Vendor Specific Protocol iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 1 Transfer Type Isochronous Synch Type None Usage Type Data wMaxPacketSize 0x1400 3x 1024 bytes bInterval 1 Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 0 bAlternateSetting 2 bNumEndpoints 1 bInterfaceClass 255 Vendor Specific Class bInterfaceSubClass 255 Vendor Specific Subclass bInterfaceProtocol 255 Vendor Specific Protocol iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 1 Transfer Type Isochronous Synch Type None Usage Type Data wMaxPacketSize 0x0400 1x 1024 bytes bInterval 1 Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 0 bAlternateSetting 3 bNumEndpoints 1 bInterfaceClass 255 Vendor Specific Class bInterfaceSubClass 255 Vendor Specific Subclass bInterfaceProtocol 255 Vendor Specific Protocol iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 2 Transfer Type Bulk Synch Type None Usage Type Data wMaxPacketSize 0x0200 1x 512 bytes bInterval 0 Device Qualifier (for other device speed): bLength 10 bDescriptorType 6 bcdUSB 2.00 bDeviceClass 255 Vendor Specific Class bDeviceSubClass 255 Vendor Specific Subclass bDeviceProtocol 255 Vendor Specific Protocol bMaxPacketSize0 64 bNumConfigurations 1 Device Status: 0x0000 (Bus Powered)

RF-tuner

RF tuner used for that design is Mirics MSi001. It supports both Zero-IF and Low-IF, which is very unusual nowadays as most tuners are just Zero-IF only.

Tuner supports multiple bands and wide range frequencies, starting from the 150 kHz and ending up to 2GHz. There is quite big gap between 260 - 410 MHz, but more bigger problem with devices planned for TV usage is unconnected LW/MW/SW and L-Band antennas. These unconnected antennas limits actual frequencies to 50 - 960 MHz. Modding should be possible, but how hard?

testing tuner limits using RF generator

Here is the measured frequency ranges from two different devices I have.

device / mode	Mirics MSi3101 SDR Dongle	Hauppauge WinTV 133559 LF
VHF_MODE	52 - 132 MHz	49 - 130 MHz
B3_MODE	103 - 263 MHz	98 - 259 MHz
B45_MODE	413 - 960 MHz	391 - 960 MHz

Staging quality

Developing that driver has been relatively big task for me. I started to looking it around one year ago but most of the implementation was done during this summer. There has been numerous new issues to resolve as that is first Linux Kernel USB SDR receiver driver. The fact is that my backgrounds are from the digital television side, DVB API, and that driver is implemented to V4L2 API. I didn't has any existing knowledge from V4L2, which put me position I was forced to learn lot of things. Now I can understand again why some "simple" DVB things could be so hard to implement for firstcomers :)

As there is still multiple unresolved issues that driver could not be mainlined yet. Biggest problem is no less than missing API. So I am putting driver to the Kernel staging which is right place to problematic stuff like that. That is something I will try to kick Mauro to get help as I have so little experience about V4L2 internals. Actually Mauro has already provided some SDR API related patches that are waiting for some discussion and review.

I collected Kernel SDR API requirement specification, which the implementation should follow in the mean it is reasonable possible to implement.

Here is the list of know issues, from critical to less critical. I think resolving API issues is enough to move that driver out from the staging.

• Kernel SDR API
• new V4L2 controls
• do not abuse video-device
• stream conversions (move to libv4l2?)
• implementation performance problems are ruled out
• GNU Radio plugin
• has performance problems (read => mmap)
• add more functionality
• stream conversions with the help of libv4l2
• separate USB ADC and RF-tuner logic to own drivers
• USB ADC main driver "msi2500"
• RF tuner to v4l2 subdev driver "msi001"
• tons of other changes!
• feel free to contribute I am open for any help
• hack hack hack!

Kernel driver development tree

I will keep latest Kernel driver development tree:
http://git.linuxtv.org/anttip/media_tree.git/shortlog/refs/heads/mirics

GNU Radio use example

Install gr-kernel GNU Radio plugin:
http://git.linuxtv.org/anttip/gr-kernel.git

Use LibV4L2 Source it provides.

gr-kernel, LibV4L2 Source, FM radio

v4l2-ctl use example

Whilst LibV4L2 Source in gr-kenel package is recommended application to use device, it is still possible to use it via standard v4l2-ctl tool. These commands will surely change without a any notification when API issues are implemented correctly, but at the time that blog post is written you could capture 94.3 MHz FM radio dump to file using following commands:

$ v4l2-ctl -d /dev/video0 --all
$ v4l2-ctl -d /dev/video0 --set-ctrl=sampling_rate=2000000
$ v4l2-ctl -d /dev/video0 --set-ctrl=tuner_bw=200000
$ v4l2-ctl -d /dev/video0 --set-ctrl=tuner_rf=94300000
$ cat /dev/video0 > ./my_fm_radio_record.bin

Pictures

generating RTL2832U driver code

Sniffing and analysing communication protocol

I decided to put online my RTL2832U reverse-engineering / hacking scripts. I get quite often questions how to sniff USB, analyse data and generate some driver code to test. I hope this example helps. Fortunately RTL2832U USB protocol is also very simple, which is easy to read and understand even with very limited existing knowledge.

I like to use native Windows XP and SniffUSB 2.0 to took USB captures. There is multiple USB sniffers available, both software and hardware. Feel free to select one you like, but be aware that these scripts only work for SniffUSB logging format. Making log format conversion scripts should be quite trivial though.

Step-by-step

1. Take sniff using SniffUSB (outputs file UsbSnoop.log). Just tune to working TV channel, one sec is enough, log file size increases very rapidly when picture is streaming.
2. Use parser.pl to convert SniffUSB log (UsbSnoop.log) more human readable form. (History: In my understanding parser.pl was a part of USBreplay toolkit.)
• ./parser.pl UsbSnoop.log > UsbSnoop.ts
3. Lets remove video data as we are not interested of it. It is only control data what we like to get.
• sed -e 's/BULK\[00081\].*$/BULK\[00081\] MPEG2 TS packet data removed/g' UsbSnoop.ts > UsbSnoop.p
4. Dump out register writes from the control data.
• ./rtl2832u.py UsbSnoop.p > UsbSnoop.c

rtl2832u.py could currently extract all registers from rtl2832u devices that has tuner e4000, fc2580, fc0013, tua9001 or r820t. There is few more RF tuners used with rtl2832u and in case of unknown tuner it could still dump out rtl2832u itself registers. Generated code could be copy & pasted directly to the Linux drivers in question and make some tests.

Bug hunting

I have found that approach most easiest way to find out and kill Linux driver bugs. Lets take for example RF tuner bug, which causes notable reduced sensitivity. Took sniffs from the working machine, generate code, copy & paste that to the suitable places of non-working driver. Rapidly that non-working driver starts working and it is time to do manual binary search to find out problematic setting and fix it. Easy? Eventually yes.

Scripts and an example (rtl2832u + fc0013)

http://palosaari.fi/linux/v4l-dvb/rtl2832u_scripts/

• parser.pl, log conversion script
• rtl2832u.py, C-code generator script
• UsbSnoop.log, SniffUSB log
• UsbSnoop.ts, parser.pl log
• UsbSnoop.p, sed log, video stream removed
• UsbSnoop.c, rtl2832u.py log, generated C-code

building USB power measurement equipment

USB power measurement

USB power measurement set-up

It is generally good to know what is consumed power all the time during USB driver development, especially because power is drained from the very limited capacity USB bus. Reliable testing of power management features  without a measurement equipment is also quite impossible.

Sometimes it is nice to compare Linux driver and Windows driver from the power consumption point of view. It is good indicator that something could be wrong if your driver is eating more power on Linux than on Windows in similar use case.  Undesirable power usage is always kinda problem, especially when device is running on battery. The other problem is heating. USB devices and chips are small and there is no heat-sinks used. Over heating could cause stability problems or even break your device in worst case.

old USB power measurement set-up

Problem with that old set-up was bad cable and cheap multimeter. Cheap unshielded cable causes communication errors with some USB-bridge chips, like Afatech AF9015. Resolution of that multimeter meter was also too low, 3.5 digit / 10A scale. Use always good quality shielded cable and multimeter which could display more than 500mA in reasonable scale.

testing wiring before cover install

Could you see the mistake in that picture? .... Meter was set to DC A whilst red cord is connected to DC uA/mA hole. Unfortunately it show quite realistic values and I installed connector cover using a glue. Later I found it shows negative numbers when meter was set correctly, which means black and red cable are cross connected. It is only visual problem as you can still connect black cable to red hole and vice versa, it was still annoying me... The other small problem I found was red and black cable were not tied together. So I decided open connector again and fix those problems eventually :-)

Pictures