running fwht codecs from userspace

The fwht codec can also be executed from userspace.

steps:

1. Load the vivid driver:

It will create two files /dev/videoX, /dev/videoY.

We will use the file with lower index to generate a picture from the vivid webcam.

Listen to a local port with netcat and dump the input to a file:

$ nc -l 8888 > from-vivid-fwht-stream.brr

Then in another terminal run:

$ v4l2-ctl -d0 --stream-mmap   --stream-to-host 0.0.0.0:8888

So now a stream with a specific format is dumped into from-vivid-fwht-stream.brr

The stream is the vivid generated video compressed as a fwht format. The compression is done in userspace.

The codeflow is:

streaming_set_cap->do_handle_cap->write_buffer_to_file -> fwht_compress

Now to do the opposite:

In the opposite direction the v4l2-ctl command listens to a port and waits.
So the v4l2-ctl should be executed first:

$ v4l2-ctl -d1 --stream-out-mmap   --stream-from-host 0.0.0.0:8888

$ cat from-vivid-fwht-stream.brr | nc 0.0.0.0 8888

And the codeflow is:

streaming_set_out->do_handle_out->fill_buffer_from_file -> fwht_decompress

So what we see is that:

1. For capture device, the v4l2-ctl  can take the captured frames, compress them with the fwht format
in userspace and then send them to some <host:port> server.

In this case, the visible_width/height should be the one that the capture device output to userspace
which are the composing on the capture if supported.

2. For output device, the v4l2-ctl can serve as a server, it listen to some <host:port>.

When a client sets a connection, it then reads a compressed fwht stream from the client and
decompress it. The visible_width/height are then the crop values of the output buffer of the output
device.

In qvidcap  there is also an fwht_decompress call. This is a case where actually no video
device driver is at all used. All it does is listen to <host:port>, read the compressed frames from
a client, decompress them and show them. So in that case there is no crop/compose values involved.
So the visible_width/height is the same as the coded_width/height.

So,

terminal 1:

dafna@ubuntu:~/clean-v4l-utils$ ./utils/qvidcap/qvidcap --port=8888

terminal 2:

cat from-vivid-fwht-stream.brr2 | nc 0.0.0.0 8888

Week 7 - Task for the week - Modifying Expectations

Here is the original very general plan from Hans:

1) Add support to vicodec to compress formats whose resolution does not
   align with the macroblock size (8x8). An example of that is 1440x900.

   This will require some work in the fwht codec and support for
   selection rectangles needs to be added to the v4l2 API.

2) v4l2-compliance: add code to detect codecs based on the M2M(_MPLANE)
   capability and which side (CAPTURE or OUTPUT) has only compressed
   formats. This is needed so test code can be added specifically
   for codecs.

3) Add missing features of the stateful codec specification, specifically
   stopping and enumerating formats (dependency of the raw formats of the
   chosen compressed format).

   v4l2-compliance has to be extended with the corresponding tests.

4) Add support for mid-stream resolution changes to vicodec, again including
   v4l2-compliance tests.

   Depending on how long 1-3 take, this can be postponed to the end of the
   internship.

5) Add stateless codec support to vicodec + v4l2-compliance tests. This will
   probably be subdivided into smaller pieces as we get to it.

6) Create libva layer for this stateless codec so it can be used by ffmpeg etc.

So items 1-3 should be done first, item 4 can be rescheduled depending on where
we are timewise.

Items 5 and 6 are very high-level at the moment, and I hope I can give a more
detailed schedule by the time you are ready to work on that.

Note that it would not surprise me if the work on v4l2-compliance will takelonger than making the vicodec changes. Testing is hard work.

What happened eventually:

Out of the first 4 tasks, I actually didn't do tasks 2,3 only 1,4.
Tasks 1 seemed to me very simple, I was sure it would take me just a few days.
Eventually it took much longer, first version was sent 16 days after the internship started.
Making the alignment correctly took time and also understanding the selection API was not obvious.
Task 4 - This one was as well not that easy and took me a while. It needed quite a lot of new code in both kernel and user space.
Task 5 is the main task of the internship, I started to work on it at Jan 21 and I'm still working on it at the time of writing this post. I already sent one version of patch sets for the kernel and the userspace.
Hans did not assign specific deadlines to the tasks as it was hard to predict how long the tasks would take me. I was mostly too optimistic regarding task 1. I thought that all is needed is to round the dimensions and pad the edge of the image with zeros, but it turned out to be more complicated.
I have 20 days until the end of the internship and I hope to finish this task by then or before:)