Showing posts with label GSoC. Show all posts
Showing posts with label GSoC. Show all posts

2012-08-22

My GSoC statistics

It is over now.

Wow, it is over finally. I have to say I was a little bit bored and stressed at the end. Whilst I know almost everything about DVB USB devices, I wasn't very familiar with DVB-core/frontend internals and quite zero knowledge from the V4L interface. Learning new things is something you are doing all the time during Kernel development, but now there was extremely a lot of new things to resolve in long period which was hard. I did some basic DVB driver hacking yesterday, after GSoC was over, and I enjoyed it. It was so easy and funny, just took some random issue, fix it, sent patch to Linux-Media Mailing List (LMML) and continue with new one. No schedules, no plans, just random hacking!


Success and failures

There is clearly one success over the all - DVB USB v2 framework. It is very important for me as I am author and maintainer of remarkably many DVB USB drivers. It was the boost of all that as I was quite pissed off limitations and bugs of old implementation.
Secondly I will list power-management, suspend and resume both DVB-core and DVB USB v2. For DVB USB v2 it is quite perfect and if problems are found there is at least one person who is willing to fix :)
Thirdly I will list failure of SDR API. My initial plan was to add SDR for DVB API, but it was refused and pointed out that V4L2 API is more correct. Without zero knowledge of V4L API I eventually ended giving it up.
The rest changes were quite small, two new DVB API parameters, some random hacks, etc.



Statistics

I collected some statistics, from Git and from my mailbox. Git statistics are collected using git rev-list command:
git rev-list HEAD --after="2012-04-24" --before="2012-05-20" ...
git rev-list HEAD --after="2012-05-21" --before="2012-08-20" ...


Fields in table below are: first number is from community bonding period (2012-04-24 - 2012-05-20) and second number is done during actual Google Summer of Code period (2012-05-21 - 2012-08-20). Consider community bonding period numbers something like what I usually do with my free-time.

Those tags are normal tags used for Kernel Git change-logs and quite self explaining at least for Kernel developers :)



commit tag community bonding period
2012-04-24 - 2012-05-20
Google Summer of Code period
2012-05-21 - 2012-08-20
Author: 18 147
Acked-by: 1 14
Reviewed-by: 0 11
Tested-by: 0 3
Reported-by: 0 1
non-authoritative Signed-off-by: 7 7


Communication during development is mainly happening on mailing lists and IRC. I read daily quite much mails coming from different lists, no idea even count. I think count of sent mails is more better metric instead. There seems to be 42 sent mails during bonding period and 145 mails during GSoC itself. Those numbers are real mails sent by hand, excluding patch mails sent using Git.

List of patches

Here are the topic of 147 patches I made during actual GSoC period. Few of those are just maintaining changes / fixes for my existing drivers I have to do over the time. Those which are marked as [media] are already merged for the Kernel, last 17 are requested but still waiting.

This is the set I will return.


2012-08-15

GSoC is over soon

DVB-frontend statistic IOCTLs

As I mentioned earlier there is some IOCTLs to request signal statistic from the device. Namely those IOCTLs are:
  • FE_READ_STATUS
    • read status flags (am I correctly tuned to channel)
  • FE_READ_BER
    • read  bit error rate
  • FE_READ_SIGNAL_STRENGTH
    • read signal strength
  • FE_READ_SNR
    • read signal / noise ratio (signal quality)
  • FE_READ_UNCORRECTED_BLOCKS
    •  read broken packets (errors seen in picture)
All of those are valid on certain conditions but not always. For example FE_READ_UNCORRECTED_BLOCKS is valid only when demodulator is locked to television channel and picture is streaming. When device is sleeping it cannot perform any of those and garbage was returned or in worst case even I/O error is raised as chip is unable to communicate.

My plan was first to add some logic to allow IOCTLs according to demodulator status flags (same as FE_READ_STATUS). After some study I found it is quite impossible as current drivers does not report all flags correctly - many times it is just all or nothing. No lock at all or all flags which means demodulator is fully locked and streaming. Due to that I ended up slightly simpler solution, just allow those IOCTLs when frontend is running (not sleeping).

All of those mentioned IOCTLs are optional. Average hardware could report quite likely almost 100% all of those but not always. On the other hand driver writers are lazy or they don't have all the needed information which leads to situation IOCTLs are not implemented. When IOCTL is not implemented error status ENOTTY should be reported. I changed error code to meet that  practice too.


DVB-frontend suspend / resume / power-management

That was maybe third time I am back with suspend / resume / power-management issues during that summer. Earlier I implemented it highly as a part of DVB-USB -framework (that I rewrote largely earlier this summer). Anyhow, logically some suspend / resume functionality belongs more for common DVB-frontend part than routines which are only for DVB-USB devices. So I split relevant parts from the DVB-USB and moved those to the DVB-frontend that all DVB device drivers could take advantage.

These are new routines offered by DVB-frontend / DVB-core:
int dvb_frontend_suspend(struct dvb_frontend *fe)
int dvb_frontend_resume(struct dvb_frontend *fe)

DVB-USB reset_resume

There was also one USB device specific issue which seems to be now resolved. USB driver model offers 3 callbacks for suspend / resume.
  • suspend
  • resume
  • reset_resume
The extra and problematic one is reset_resume. It is quite similar than normal resume but for some reason (like BIOS implementation / bug) USB has reset during suspend/resume. Fortunately my motherboard has extra USB3.0 controller which did reset_resume whilst the USB2.0 controllers integrated to south-bridge did normal resume. Because of that I was able to test both cases and find out proper implementation.

Common DTV standard parameter validation

There is quite many possible parameters which could be programmed to frontend in order to tune to television channel. Parameters are defined in different standards. All values are not valid for every standard and also different partly parameters are used. I have seen it quite hard to validate parameters and allowed combinations inside device driver. That makes me wonder if some common routines per standard are useful...
After all I added few routines for most common standards, which I was familiar. Those are DVB-T, DVB-T2, DVB-C (Annex A) and DTMB. I didn't added more as it was very time consuming to find out needed information from many specifications. For example DVB-C, there was at least three different papers to look:
  • EN 300 429
    • European Telecommunications Standards Institute
    • main specification
  • ITU-T J.83
    • The International Telecommunication Union
    • some additions (Annex A/B/C)
  • NorDig Unified Test specification
    • NorDig
    • real life requirements for digital televisions
    • for example frequency limits

Working with that task I take note that these are not listed properly even Kernel documentation. Kernel documentation is place where application developers will likely try to find out possible parameters at the very first. There is clear need to document these correctly...


2012-08-08

DVB USB v2 on the road!

DVB USB v2 merged to Linux Media master

Beginning from the last week there has been finally some progression about DVB USB v2 stuff I did earlier. It missed Kernel 3.6 schedule but has now merged the Linux Media master which means it will be eventually Kernel 3.7. Mauro Carvalho Chehab reviews it last week and returned back for me as it failed to compile with allyesconfig. I wasn't even aware such configuration target as I build using Fedora's default configuration, which is quite everything as a module. Breaking build is quite no-no because of Git bisect. I was forced to made huge Git rebase operation in order to keep bisect functional. It was set of over 100 patches and I fixed maybe 20 merge conflicts during that rebase. Guess how much wasted time... Positive point is that now I am quite familiar with Git rebase. It is tool that most developers are forced to deal at some point when fixing merge conflicts or some other issues.

After I was fixed allyesconfig problems and ensured it builds after every single patch I resend PULL-request and it was finally accepted. Mauro converted dvb_usb_az6007 driver for that new framework and dvb_usb_lmedm04 from Malcolm Priestley is under the review. I expect to see more drivers soon, because there is some notable problems with old implementation.

DVB-core situation

Time is running quickly and I have only about one week to go with GSoC.

Much is done but much is still to do. SDR API was one thing I was most interested, but unfortunately it goes a little bit different direction than I was first thinking. I was adding SDR support for DVB API but instead wasted time to learn V4L2 API. Have to give it up for now, but likely I am back this topic later this autumn...

Looking the initial list there is still one topic I see important - general validity checkings for frontend side. I will spend rest development time for that, unless something urgent and time-consuming appears from DVB USB side.

2012-07-31

DVB power-management on suspend

Back to the power-management again

Last week I finished with DVB power-management, the issue I touched on earlier too when implementing general suspend / resume support for DVB USB. For now I looked how to implement real hardware power saving functionalities when device is suspended. Initial implementation was lacking all the power management, allowing only computer to enter suspend when television was still streaming. All the chips were still left full power state and thus continued draining full current which is simply against Kernel power-management rules.


In order to fix that problem nicely and general level enough I was forced to study issue from multiple perspectives. It is clear that DVB devices are much more complex than usual Kernel drivers. That is coming from the fact there is at least three sub-drivers (USB/PCI-interface, demodulator, RF-tuner) and each of those works quite independent. Fortunately there is existing power-management callbacks (named init and sleep) which are even often implemented. You can consider those equivalent to USB -framework PM callbacks that are named suspend, resume (and reset_resume). Another lucky point here is that DVB-frontend has cache structure, which basically stores all the transmissions parameters used currently. So as we has existing init/sleep callbacks for power-management and all transmission parameters stored, it should be relatively easy to recover device at the state where it was.
  • on suspend
    • stop remote controller polling (interface driver)
    • stop ongoing stream (interface driver)
    • sleep demodulator (demodulator driver)
    • sleep tuner (RF-tuner driver)
    • sleep interface (interface driver)
    • return with success to inform Kernel PM device is suspended
  • on resume
    • wake-up interface (interface driver)
    • wake-up tuner (RF-tuner driver)
    • wake-up demodulator (demodulator driver)
    • acquire DVB-frontend to retune (DVB-core driver)
    • resume ongoing stream (interface driver)
    • resume remote controller polling (interface driver)
    • return with success to inform Kernel PM device is resumed

Simple? Actually it is, due to no need to touch existing demodulator and tuner drivers. And seems to work nicely. What I tested with few USB devices using VLC, all those continued to show same TV-channel just after resumed!

I still have some thoughts to make it more general by moving that functionality partly to the DVB-frontend. Currently process is lead by DVB USB core parts and DVB-frontend is only asked to perform retune. It could be wise to add suspend and resume functions to the DVB-frontend and let the DVB-frontend then call demodulator and tuner driver init & sleep respectively.

sub-driver level power-management issues

During the development I made simple USB power meter in order to see some real life statistics about device current usage. Quick testing with few USB devices reveals we have quite much room for improvements in that area. I will go back to that topic soon when new multimeter I ordered arrives.

device draining around 280mA when in use


2012-07-20

learning learning and GPIOs...

Partytime!

I kept extra long weekend holiday, wow! Reason for that was Ilosaarirock festival I attended with friends. Heh, we were there from Friday to Sunday night and back to home late Monday... not surprise I was a little bit tired. There was very good reason to party as my Google Summer of Code project (that Kernel Media project for Linux Foundation) Midterm Evaluations successfully were passed!

GPIOs

GPIO, shorthand for General Purpose Input/Output

implement functionality of LNA API call in real life

I was forced to find out "proper" solution to control LNA (inside Kernel driver) after I added DVB API support for it. Very often those LNAs are controlled by GPIO line or two. Almost every chip has few GPIOs, for DVB -device this means there could be GPIOs on USB/PCI-bridge, demodulator and tuner. LNA commonly is very simple to control, ON or OFF, which means there is no own driver for LNA chip. Controlling this kind of general device specific properties belongs to the bridge driver.

LNA control path

I should write short article about anatomy of DVB-device but as there is no such article yet, lets explains some wiring as background. I used PCTV nanoStick T2 290e during development, but for most others it is just similar. That device LNA is controlled by demodulator GPIO. Here is the control path, from computer to LNA:

PC|bus=USB|chip=bridge|bus=I2C|chip=demodulator|bus=GPIO|chip=LNA

LNA controlling problem

The problem. Bridge wants to set LNA, but LNA is controlled by the demodulator. How to deliver that request to the demodulator driver? Simplest solution is to export function from demodulator driver to perform request. Function like set_lna() or set_gpio() and call that from the bridge driver. Yes, works, used widely, easy to implement, but so home-brew.

Kernel gpiolib framework

After digging Kernel code and documentation I found there is existing GPIO support. I decided to  try that. At the first glance it looked like only for the motherboard GPIO handling and my need is inside device driver. Looking code more I found it was extended to handle GPIO expanders using gpiolib.  This kind of DVB -device could be seen actually as a USB-I2C-GPIO-expander. For the bonus point we get userspace sysfs interface for free! It is very nice to have as it provides easy access for GPIOs during development.

gpiolib sysfs example

For example this is how I can enable (echo 0) and disable (echo 1) LNA in question:
echo 253 > /sys/class/gpio/export
echo 0 > /sys/class/gpio/gpio253/value
echo 1 > /sys/class/gpio/gpio253/value
echo 253 > /sys/class/gpio/unexport


Final notes

I encourage all the other DVB driver developers use same solution for demodulator and tuner GPIOs. It is now learned and found to working at least some level. Implementation seems to be quite simple and interface is standard provided by Kernel gpiolib framework. Look example from the cxd2820r demodulator driver.

2012-07-12

[PATCH RFC] add LNA support for DVB API

LNA (Low-noise amplifier)

LNA is small antenna signal amplifier inside DVB device. Usually it is external component located between antenna connector and RF-tuner. There is usually amplifiers between whole signal path but LNAs and those which are inside of device. Normal LNA we have in digital television device is simple on/off. When it is off it bypass signal, causing still some loss, and when it is on it amplifies signal using some static factor. That factor is called gain and unit used usually is Decibel (dB).
There is also LNAs offering more than one gain level. For example external DVB-T/C LNA NXP BGU7033. It does have 3 modes, 10 dB, 5 dB and bypass -2 dB.

LNAs having even more adjustable gain levels are called variable gain amplifier (VGA). Also there is LNAs/VGAs integrated to the RF-tuner... But I am not going to go deeper these two cases as main need for that LNA API addition is simple external LNAs between RF-tuner and antenna connector.


comments for DVB LNA API


[PATCH RFC] add LNA support for DVB API


2012-07-11

DTMB support for DVB API

V4L2 SDR API

It has been rather slow progress week. Initially I started studying changes needed to support SDR (Software Defined Radio) via Kernel API. For me it sounds quite easy and logical to extend DVB API for SDR. Just adding new delivery system and implement other relative small changes for DVB-core. After some discussion on Linux-Media mailing list, idea of extending DVB API was refused as V4L2 API is more correct API for that kind of stuff. V4L2 API covers FM-radio, camera, web-camera and analogue television stuff whilst DVB API is used only for digital television. Unfortunately I have quite zero experience from V4L2 devices. After hacking with Silicon Laboratories Si472x based USB FM-radio I decided to leave whole SDR for a while. Have to get more suitable radio for V4L2 learning...

DTMB support for DVB API

After adversity with the SDR I decided to do easier DVB enhancements I planned. DTMB (Digital Terrestrial Multimedia Broadcast) is Chinese digital television standard and there is some existing Linux Kernel drivers for that standard. One of those drivers, HDIC HD29L2, was made by me in last autumn and initial interest is coming from there. Big work for whole DTMB API support was done during last Christmas holiday. I even ran some initial RFCs but never finalized work. Learning many research papers and listing all the used parameters, finding out equivalent ones from the existing API, was quite many working days in that time!


1. [PATCH RFCv3] add DTMB support for DVB API
2. [PATCH RFCv3] add DTMB support for DVB API

2012-07-04

PULL-requests for Kernel 3.6

[GIT PULL FOR v3.6] DVB USB v2

I pull requested all the DVB USB changes I did. Quite big patch bomb, 103 change-sets! Those are requested for next possible Kernel 3.6, but it could be delayed to 3.7 if some issues are raised. No comments so far...

[GIT PULL FOR v3.6] DVB USB v2


[GIT PULL FOR v3.6] rtl2832 and misc changes

2nd set I pull requested for the same Kernel was some maintenance stuff, bug fixes and minor enhancements to my existing drivers. I put also a long-awaited RTL2832 DVB-T demodulator driver by Thomas Mair to that set too as I did much work for reviewing it and it does have close relation to my dvb_usb_rtl28xxu driver. Hopefully it gets merged at that time...

[GIT PULL FOR v3.6] rtl2832 and misc changes


2012-06-29

DVB core enhancements - comments please?

As you could remember I initially scheduled my GSoC project to two phases. First one was DVB USB enhancements and second part was for DVB core issues. DVB USB changes are quite ready and I am going to sent upstream merge request in next few days. I will write article about that later in next few days.

I did some planning for the DVB core part recently. I ended-up listing known bugs, insufficient functionality and some new features and sent plan to the Linux-Media mailing list for the comments. No comments so far for that mail - but many of those are discussed earlier too.

DVB core enhancements - comments please?


DVB core enhancements - comments please?

Here is my list of needed DVB core related changes. Feel free to comment - what are not needed or what you would like to see instead. I will try to implement what I can (and what I like most interesting .

general validly checking for demodulator callback input values

  • currently each driver needs to validate those
  • values are highly hooked up to used television standard
  • we can do almost all validly checking inside core
  • we can also check if call is possible to perform in given condition
  • for example BER is not valid when demod is unlocked

suspend / resume support

  • support is currently quite missing, all what is done is on interface drivers
  • needs power management
  • streaming makes it hard
  • quite a lot work to get it working in case of streaming is ongoing

use Kernel power management instead of own

  • there seems to be Kernel services for power-management
  • study if it is wise to use Kernel services instead of own
  • own PM is still working very well, at least I dont know any problems

SDR - Softaware Defined Radio support DVB API

  • http://comments.gmane.org/gmane.linux.drivers.video-input-infrastructure/44461
  • there is existing devices that are SDR (RTL2832U "rtl-sdr")
  • SDR is quite near what is digital TV streaming
  • study what is needed
  • new delivery system for frontend API called SDR?
  • some core changes needed, like status (is locked etc)
  • how about demuxer?
  • stream conversion, inside Kernel?
  • what are new parameters needed for DVB API?

DTMB standard support for DVB API

  • it is Chinese DTV standard
  • I already ran RFC but have been too busy for implementing it :]

LNA (low-noise amplifier) support for DVB API

  • there is quite a lot of devices having LNA
  • currently not supported => LNA is configured off typically

offer polling method for statistics

  • many static counters could not be read as a "one go"
  • typical cycle is : start measurement => wait => read counters
  • some drivers starts own internal work-queue for polling (complexity)
  • some drivers blocks IOCTL when taking measurement (bad)

fix frontend properties

  • those has been broken since MFE => SFE change
  • currently implemented as a properties per driver
  • need to be properties per delivery system
  • are broken because driver/chip could support multiple DTV standard

2012-06-08

AF9015 driver converted to new DVB USB

AF9015 driver converted!

I finally decided to publish one rather complex DVB USB driver as a demonstration what it looks like. I selected Afatech AF9015 driver as it is one of the most complex and main motivator of whole DVB USB changes. I have also Anysee, AU6610 and EC168 drivers converted, but those are far away complexity compared for AF9015, since not so interesting.

Code size reduced a lot

All-in-all, code size goes down rather much. It was 2084 LOC and now it is 1609 LOC - it is near 500 LOC less. Basically it is all coming from the removed hacks. Now all needed functionality is generalized to the common DVB USB and ugly hacks are removed from the individual driver.

PATCH: af9015: switch to new DVB-USB

Near future plans

I am quite happy for features common DVB USB now supports. I think all but dynamic USB ID are implemented somehow for those I initially planned. There is still much work to do when implementing all correctly and test error paths. There is still at one big technical problem to solve. It is that delayed init which was added to fix firmware loading. I can get it crashing rather easily just repeatedly loading and unloading DVB USB driver... I suspect it is coming from the fact both workqueue running delayed device intialization and USB-core end up same routines. For example workqueue is downloading firmware and at the same time module is unloaded by user. Likely some locking is needed to prevent module unload in that case.

2012-06-05

Dynamic USB device ID

Dynamic USB device ID


Reference designs

Nowadays many DVB UDB devices are actually chip vendor reference designs which means those are very similar. Used chips are same and wired together similarly, meaning only new device id is needed for the driver in order to get it working.

It takes too long

There is always significant delay from the device release to the point support is arrived for the distribution. That delay is coming from the different schedules. First it takes time until driver author gets info about new device is needed to add driver. He has to find out hardware or at least someone who can test patch. After that there is Kernel schedule. Kernel has own merge window and release candidate cycle that takes many months. In my understanding simple USB IDs are allowed to add driver during release candidate phase, but usually it still goes through merge window which adds significant delay. And finally there distribution schedule. It is usually needed to upgrade used distribution as they do not upgrade new Kernel version during release cycle. All-in-all, it could take year or so.


Promote the device ID to driver


There is feature called dynamic USB device ID to tackle that delay. User can tell for the Kernel that he wants try if given device driver could drive his new device. For example we could say load dvb_usb_af9015 driver for the device having USB ID 15a4:9016:
echo 15a4 9016 > /sys/bus/usb/drivers/dvb_usb_af9015/new_id


DVB USB and dynamic IDs

Currently DVB USB doesn't support dynamic IDs. Today I did some work in order to add support for it. Unfortunately I did not find out as nice solution as I was hoping. As for now I use .driver_info field from the struct usb_device_id to pass all needed to data to the DVB USB. In practice .driver_info field carries pointer to the struct dvb_usb_device_properties. In normal case all needed data is inside MODULE_DEVICE_TABLE() but in case of dynamic ID there is no entry inside MODULE_DEVICE_TABLE() and thus no needed pointer inside .driver_info.

Solution is to implement own .probe() and set .driver_info for dynamic ID. And finally pass that all to the DVB USB. Not very ugly, nor nice.

What I would like to see is dynamic ID entry which could be added to the driver MODULE_DEVICE_TABLE() similarly as others. Maybe there is even some reserved USB vendor IDs for special purposes that could be used.

struct usb_device_id {
    .idVendor = VENDOR_ID_DYNAMIC
    .idProduct = PRODUCT_ID_DYNAMIC
    .driver_info = <own data>
}

2012-06-04

DVB USB firmware download problems

DVB USB firmware download problems


suspend / resume

I listed fixing resume from the suspend as a one topic of the DVB USB to fix. There has been very long time nasty suspend / resume bugs which seems to be related for the firmware downloading. Reason has not been clear thus those were never fixed. I found one bug report back from the 2009 which was sent for me, as my af9015 driver crashes during resume. At that time I didn't have much experience how to hunt bug like that, which was out of my chip driver. I made simplest possible DVB USB driver which just requests firmware download - and crashes - just as a proof of concept. Unfortunately I did not get any help and it left unresolved.
Crash I made 2009: DVB USB resume from suspend crash

Fedora 17 installation to the SSD

As I got very many Kernel crashes during the development and every crash means booting which wastes time around 10 minutes per boot. Also Fedora 17 was just released and I wanted to install it, so I decided to buy new SSD and install Fedora 17! Install goes nicely, taking only minute or two, it is unbelievable how fast SSD is. I have to say hate Gnome3 UI, thus I was earlier running it "fallback" mode but now I decided to give opportunity for Cinnamon.

DVB USB devices on Fedora 17

Finally I started to test how DVB devices were working. It came as a small surprise that plugging DVB USB stick in hangs device around 30 seconds until it appeared. Looked the logs and there was udev errors. It was not totally broken as it still started working but takes 30 seconds and it happened for every DVB USB device which uses firmware.


Jun  1 19:24:24 localhost kernel: [   59.820552] dvb-usb: found a 'Afatech AF9015 DVB-T USB2.0 stick' in cold state, will try to load a firmware
Jun  1 19:24:54 localhost udevd[423]: worker [1581] timeout, kill it
Jun  1 19:24:54 localhost kernel: [   89.853168] dvb-usb: downloading firmware from file 'dvb-usb-af9015.fw'

udev and blocking firmware download at module_init()

Clear suspicion was udev as there was nothing changed inside DVB USB which could break it like that. I downgraded udev to Fedora 16 version and it started working. As I was not any familiar with the udev I decided to open ticket for the Fedora Bugzilla targeting it udev. It didn't take many minutes Kay Sievers replied to Bugzilla, explaining it is Kernel driver issue as it blocks module_init() by downloading firmware. The reason it now arises is udev. Its behavior was changed and we wasn't aware that it was needed to change Kernel drivers not to block module_init().
Fedora Bugzilla: DVB USB device firmware requested in module_init()

The fix

I implemented non-blocking module_init() / USB probe() for DVB USB and problem disappeared. It was done by delaying old probe functionality, which downloads firmware too, using Kernel workqueue and returning always success for the probe() / module_init(). Doing it that way is not prettiest option at least for my mind but there is no any better I am aware. Downside here is that we must deregister device in case of something fails during "old probe". I did that de-registering using usb_driver_release_interface().

suspend / resume fixed too?

I was unable to reproduce old, and very well known, suspend / resume bug with my current hardware and software. Anyhow, it is quite likely coming from the same blocking firmware download issue. Resume from the suspend behaves quite similarly as plugging DVB USB stick to the USB port.

There is still another existing suspend bug. If you are streaming DVB device, like watching television, it will never go to the sleep. It just hangs. That is DVB core issue as in my understanding there should be some mechanism to stop stream before going to sleep. I will fix that later this sumer after DVB USB issues are fixed.

2012-05-31

USB streaming buffers


USB streaming buffers

DVB USB allocates rather big chunks of coherent memory buffers for the USB streaming at the start and frees those on disconnect. One buffer is reserved for each used USB URB, which is around 5-7 buffers as there is that many URBs used generally. After the multi-frontend (MFE) implementation own USB streaming buffers were allocated for every frontend. Currently there is three frontends registered for the single adapter in worst case. And what is the problem: only one frontend can be used at the same time and we allocate buffers for all the three!

I resolved problem allocating biggest needed buffer at the start and changed all frontends in given adapter to use that. URB configuration is performed dynamically to fit needed stream. Quite lightweight solution. Maybe there is still room for the improvement as it allocs and frees URBs every time when streaming configuration is changed. Allocating and freeing few URBs is very cheap operation but it could be avoided unless USB endpoint type is changed (bulk or isochronous). And after all use case where frontend is changed all the time is very unusual. User just choose one frontend, which offer needed television standard, and uses that. So it must be fine :)

What is expensive in whole subject is coherent USB transfer buffer and now it is used 2/3 less than earlier. Someone would like to ask why not to alloc and frees all the memory, URBs and buffers, every time when streaming is started and stopped. There is very good reason to avoid continuous allocs and frees as memory goes fragmented and finally allocation of big coherent chunks of buffer memory will fail. That is realized few times in the recent future as some other DVB devices are stopped working. Limited memory is especially problem of embedded devices.  Believe or not, it is very common DVB sticks are used on embedded routers or STBs.

Related links

[PATCH] em28xx: pre-allocate DVB isoc transfer buffers
http://comments.gmane.org/gmane.linux.drivers.video-input-infrastructure/44570

DVB USB: change USB stream settings dynamically
http://comments.gmane.org/gmane.linux.drivers.video-input-infrastructure/48861

2012-05-27

GSoC week 21.


Week 21 - planning

 

First week

First week is almost over. It is almost over since I am still going to do some USB streaming buffer tests tonight :-)

This week was scheduled as a RFC and planning for DVB USB changes. DVB USB means those general routines used by almost every digital television USB stick. I think it could be called as a sub-driver, framework, library or even DVB USB core. I still like to call it just DVB USB or DVB USB framework.

I did early start for the first week and sent two RFCs describing planned changes 1-2 weeks earlier. It was not a big surprise there was no many comments. Due to that I think changes I was introducing are not totally wrong :)

Implementing

As there was no much discussion about introduced changes I started implementing those. I copied current DVB USB implementation as a starting point, leaving old implantation still alive as a parallel. It is almost impossible to make such big changes for the existing one as there is about 30 individual DVB USB interface drivers using DVB USB core routines. It is still unknown how to make switch-over for the new implementation. I see two possibilities; 1) move driver by driver to the new 2) move all changes one by one to old implementation.

During the development I converted and tested all the time three of my drivers against new routines. Those drivers were au6610, ec168 and af9015. af9015 is one of the most complex DVB USB device drivers and full of hacks. Problems with the af9015 has been my main motivator for most of these DVB USB changes I planned.

Some improved numbers


Changes what I did this week removed many hacks from the af9015. Also those reduced binary size of the af9015 very much.

      text   data  bss    dec   hex  filename
old  14878  13232   40  28150  6df6  dvb-usb-af9015.ko
new  14575   3952    8  18535  4867  dvb-usb-af9015.ko

Space saving is coming from the new way of passing data to the DVB USB framework. Earlier all device data was put inside static struct as now it is passed using driver_info field of the struct usb_device_id.

Size of that configuration structure was always constant, no matter if there was one or 12 devices as current limit is. af9015 driver contains 3 of those big configurations structures. There is some drivers having ~30 similar structs - guess how much space is wasted?

Related links

[RFCv1] DVB-USB improvements
http://comments.gmane.org/gmane.linux.drivers.video-input-infrastructure/47994

[RFCv1] DVB-USB improvements [alternative 2]
http://comments.gmane.org/gmane.linux.drivers.video-input-infrastructure/48572

My DVB USB development tree
http://git.linuxtv.org/anttip/media_tree.git/shortlog/refs/heads/dvb-usb

Blog is open!

About the Blog 

 

Google Summer of Code 2012 attendee!

I have been planning own blog for my Kernel development stuff for a while but newer got enough time nor motivation to learn things to start blogging. Things gone ahead after I was chosen for the Google Summer of Code programme as there was some pressure for the blog to report progression of the project.

My GSoC project is under the Linux Foundation umbrella and it is Kernel development for my favourite area - Linux television interface. Traditionally I have been coding only new television chip drivers. Type of those chip are; USB-interface (aka DVB USB bridge), demodulator and RF tuner. There is around 20 chip drivers currently I have done which is huge amount of drivers for the single developer.

Current problems

During the development of those drivers I have gained very good overall understanding of the whole digital television side of the Linux Kernel and also that technology in general. It is not big surprise I have met a quite lot general issues, issues that are outside of individual driver, somewhere in our core routines. Those issues and limitations are raised up during a long time as hardware has gone more complex and old routines are not flexible enough for the new hardware.

Due to those core limitations there is many ugly hacks needed for the drivers in order to operate. Sometimes those hacks are even very wrong but still mainly working with a little bit luck. More often those hacks still means "only" duplicated code, complex code and wasted resources.

Usually I am too busy to hack support for new the hardware and due to that I have no possibility to allocate enough resources for fixing known general issues. Now that GSoC project gives me possibility to fix those!

GSoC schedule

That GSoC project is workload of three months. Initially I split it to the two big parts, DVB USB enhancements and DVB core enhancements. DVB core contains mainly DVB frontend changes.

Planned schedule is here:
week 21: DVB-USB plan driver interface + RFC
week 22: DVB-USB implementation
week 23: DVB-USB implementation
week 24: DVB-USB implementation
week 25: DVB-USB implementation
week 26: DVB-USB convert few existing drivers to use that new interface
week 27: DVB-CORE learn frontend handling
week 28: DVB-CORE plan needed changes + RFC
week 29: DVB-CORE implementation
week 30: DVB-CORE implementation
week 31: DVB-CORE implementation
week 22: DVB-CORE convert few existing drivers to use that new interface
week 33: pencils down, finalize
week 34: pencils down, finalize