Showing posts with label gsoc. Show all posts
Showing posts with label gsoc. Show all posts

Tuesday, September 07, 2010

xmf/ixmf plugin updates

For the past few months I've implemented a few XMF plugins for GStreamer's GSoC. The GSoC objectives and results are here:
  • Add XMF support into GStreamer (both creation and parsing of XMF files)
  • Have a base for XMF sub types (mXMF - mobile ringtones, iXMF - games, interactivity, among others)
  • Perform iXMF experiments
  • Get to know the GStreamer community, make professional contacts, learn the technologies in question (GStreamer, GObject, GLib, etc)
  • Get in touch with the XMF/iXMF working group
I'm glad to say we managed to accomplish these objectives. From the code POV, we have the following plugins:
  • xmfmux: capable of creating XMF files
  • xmfdemux: capable of demuxing XMF files
  • xmfbin: worth-mentioning experiment in which we tested how to perform some iXMF features within the gst context.
It's important to note that me and my mentor will maintain these plugins and keep them updated according to newly published specs. Although they're in a near-stable state, we're in the process of rounding them before pushing to -bad.

Concerning the community, we've held a conference call with key members of the iXMF working group. It was a great talk and they're definitely backing up our project. I joined the group and I may attend to following conference calls.

Monday, July 19, 2010

xmfbin experiments, ixmfmux

This is the third report for the iXMF GSoC project. In this phase I've done experiments on an element called xmfbin, that would implement similar behavior of an iXMF engine. Me and my mentor have discussed the difficulties and aspects of this element for defining what we'll do next.

The experiments were all about learning more about the runtime environment required for ixmf. xmfbin was able to play simultaneously mp3 files embedded in a xmf file, created using our own xmfmux component. We had some problems when playing a mp3 file two times while using pulsesink, debug shows there may be a problem with discontinuity but alsasink did not present the same problem. We're still looking into it.

There are also other ideas regarding how ixmf playback could be done using GStreamer, we have stashed them and we'll probably revisit them on the next few weeks.

Now I'll start coding ixmfmux, which is just an extension of xmfmux. The addition is that it embeds special data structures used for ixmf effects (e.g fading, transitions, volume control, etc).

Week Date Event Status
0May 18th First component delivered xmfdemux delivered
1 May 24th GSoC officially begins Designing xmfmux
2May 31th Coding xmfmux xmfmux, unit tests
3Jun 7th Coding xmfmux muxing of complex folder structures
4Jun 14th Second component delivered xmfmux delivered, start designing ixmf playback engine and muxing requirements
5Jun 21th iXMF bin design and prototype extended
5Jun 28th xmfbin experiments xmfbin initial implementation
6Jul 5th xmfbin experiments simultaneous playback tests with formats other than (e.g. mp3)
7Jul 12th xmfbin experiments seeking, samplebin-like behavior
8Jul 16th GSoC Mid-term GSoC midterm evaluations
8Jul 19th ixmf startup start coding ixmfmux component for samples generation
Project Schedule

Monday, June 21, 2010

GSoC Project Report 2 - xmfmux

This is the second report for the iXMF GSoC project. In this phase I have created the xmfmux component, the remaining requirement for starting diving into ixmf.

Concerning GSoC timeline, here's how we're scheduled:

Week Event We're at
0 (May 18th) First component delivered xmfdemux delivered
1 (May 24th) GSoC officially begins Designing xmfmux
2 (May 31th) Coding xmfmux xmfmux, unit tests
3 (Jun 7th) Coding xmfmux muxing of complex folder structures
4 (Jun 14th) Second component delivered xmfmux delivered, start designing ixmf playback engine and muxing requirements
5 (Jun 21th) iXMF bin design and prototype In progress
Project Schedule

Important information:
  • Project repository:
  • Contact
    • Me: André Dieb Martins (andredieb on IRC-freenode) -
    • Mentor: Thiago Sousa Santos (thiagoss on IRC-freenode) -
  • XMF - introduction about the base format for iXMF
  • iXMF - introduction about interactive XMF
  • iXMF IASIG - interactive XMF working group (contains the draft spec)
  • mXMF - mobile XMF
  • Blog post by me about XMF/iXMF and the GSoC project
  • First report: xmfmux
Now to the report itself.

As you can see from our schedule above, we spent about 2-3 weeks coding the xmfmux component. The component is currently in its most basic form (doesn't support metadata yet) but it's able to fully mux xmf files and build any node structure specified (for instance, multiple folders, subfolders). This was a functional requirement for iXMF, as it specifies multiples folders for storing its different resources descriptors. Here we show some usage:

1. Single node structure, one root node that is actually the file (most common mxmf files have this layout):

filesrc location=file1.midi ! xmfmux ! filesink location=output.xmf
2. Two nodes owned by a root folder node:

filesrc location=file1.midi ! xmfmux name=muxer ! filesink location=output.xmf filesrc location=file2.midi ! muxer.
3. Subfolders:

Desired layout:
  • Root
    • Folder1
      • Folder2
        • file1
filesrc location=file1.midi name=file1 xmfmux name=muxer ! filesink location=output.xmf file1 ! muxer.Folder1:Folder2:file1
Note that in this pipeline we specified the folder structure through the pad names, folders separated with : tokens.

Now on week 5, I'm studying and designing how the ixmf bin will be implemented. Concerning the xmfmux and xmfdemux components, they still lack a couple functionalities required by ixmf:
  • xmfdemux: Pull pads for srcpads created
  • xmfmux: metadata
Week 5 will be about implementing these remaining functionalities and keep designing the ixmf bin. Next report will expose the design study results and the implementation roadmap.

Tuesday, May 18, 2010

GSoC Project Report 1 - xmfdemux

This is the first report for the iXMF GSoC project. First let's have some useful information about it:

Project repository:
  • Me: André Dieb Martins (andredieb on IRC-freenode) -
  • Mentor: Thiago Sousa Santos (thiagoss on IRC-freenode) -
iXMF related resources:
  • XMF - introduction about the base format for iXMF
  • iXMF - introduction about interactive XMF
  • iXMF IASIG - interactive XMF working group (contains the draft spec)
  • mXMF - mobile XMF
  • Blog post by me about XMF/iXMF and the GSoC project

For achieving the bigger goal (iXMF), GStreamer needs to support the base format (XMF). For a better understanding of the format, we decided to start the project writing a XMF demuxer element.

I started implementing the demuxer element about a week ago and today we played for the first time a XMF file. The demuxer is capable of parsing a mxmf file (mobile XMF - audio/mobile-xmf, bundle of two audio files: mDLS + MIDI) and creating sometimes-pads for each internal file.

It seems that GStreamer does not yet support mDLS, so the test concerned only the internal MIDI file (using wildmidi MIDI decoder plugin). The pipeline used was:

filesrc location=sample.mxmf ! xmfdemux ! wildmidi ! pulsesink

Still, there are a few TODO's for this element:
  1. Pull mode for created pads: when working in sink pull mode, it would be nice to let our created pads also work in pull mode. This can be tested with the timidity plugin which supports only pull-mode.
  2. Finish implementing file references handlers: currently we're only parsing in-line resources data. There are a few more types of reference that should be implemented before moving forward to the next element.
I'll be working on these TODO's in the next week and there should be a report concerning the next steps, which are currently under discussion with Thiago.

I'd also like to call out our fellow GStreamer developers for help, insights, suggestions, reviews, etc. It would be great to have feedback from the community.

Wednesday, April 14, 2010


This year I applied to GStreamer's GSoC proposing to add iXMF support, consequently XMF.

For those who don't know iXMF, it's an interative audio format/container. It's a low-overhead meta file format capable of bundling collections of different data resources into a single file.

It's designed minding games sounds/music, audio skins, artist autonomy, playback instructions and many other aspects.

To quote iXMF's background description:
This new file format will put artistic control into the hands of the artists, keep programmers from having to make artistic decisions, eliminate rework for porting to new platforms, and reduce production time, cost, and stress.
Sounds powerful, doesn't it?

Now back to the proposal, my initial idea is to start by adding XMF meta file support (parsers and writers elements). This would be useful for packing multiple audio files (possibly with different formats) into a single file.

Once done, the fun part jumps in. Being an interactive audio format, iXMF playback is designed to be somehow event-driven (or signal-driven), e.g. a game application signals play (shot_sound). Additionally, these packed audio resources can contain scripts that allow transitions (e.g. crossfade) and playback parameters (e.g. volume, pan, etc). For example, the shot_sound resource could specify a play cross-fade effect.

Concerning this engine, I have discussed some implementation details Thiago and Stefan,
elder GStreamer developers. I have been pointed to, which seems to be a good idea to be aggregated here. I also gathered lots of mxmf samples (mobile XMF), which can validate the parser/writer elements.

Eager as I am, I already done some research on simultaneous play of multiple audio files (with different formats) and wrote a simple element called decoderadder which does the basic job of decoding and adding the different resources. A simple test pipeline could be
gst-launch filesrc location=background.mp3 ! decoderadder name=a ! pulsesink audiotestsrc freq=100 ! a. audiotestsrc freq=200 ! a.
which plays an mp3 audio file (e.g. a game background music) and two sin waves with frequencies 100Hz and 200Hz.

Currently I'm researching how to implement seek in decoderadder and adding signals for scheduling plays / stopping resources. I'm also researching a way for caching/predecoding samples for almost-instant play (Stefan pointed me to buzztard's memoryaudiosrc element).

I must say I'm very excited about this project, so, fingers crossed!

Wednesday, August 19, 2009

GSoC final report

We've just passed the firm pencil's down date. Here's a final report for Eupnp GSoC project:
  • Discovery, Description, Control features completed (which provide a basic control point functionality set)
  • Multi-toolkit: one can easily integrate Eupnp with Qt, Glib, Ecore or any other toolkit/framework that provides the basic functionality required. We already have integration with Ecore (EFL)
  • Event Notification incomplete (we're able to subscribe for events but for now you're only able to receive raw messages)
  • Internet Gateway Device configuration tool incomplete (able to list IGD devices)

Even though the program ended, I'll obviously keep working on Eupnp. Here's an updated TODO list:
  • Finish implementing Event Notification
  • Finish IGD configuration tool
  • Finish API documentation, write a tutorial
  • libeupnp-av (UPnP Audio/Video specification) extension library
Ideas/future features:
  • Universal control point
  • Python bindings
  • Maemo porting
  • Canola plugin
  • Device building ability (UPnP server)
PS: Big thanks to Gustavo Barbieri (my mentor) who helped me throughout problems and taught me lots of programming techniques. Also thanks to EFL guys (raster, Sachiel, vtorri, cedric, etc) who had the patience to read my patches and answer my stupid questions on IRC :-P.

Monday, August 10, 2009

Eupnp & GSoC


I've just finished implementing the last requirement (embedded devices parsing) for our IGD client.

Concerning the project as a whole, we have the following features and TODO's:

  • Discovery: SSDP client
  • Description: fast and light SAX parsers for devices and services description files
  • Control: SOAP support implemented
  • Integration interface for external toolkits/libraries (ecore, glib, qt)
  • eupnp-ecore: integration library with ecore library
  • Command line control point application example
  • IGD GUI application listing devices
Missing/TODO (by order of priority):
  1. IGD GUI controlling IGD devices
  2. Event Notification: subscribing for variables updates
  3. Docs, tutorial
  4. UPnP Audio/Video extension library (libeupnp-av)*
Also, it's good to remind that today is the suggested GSoC pencil's down date. To quote the timeline:
August 10: Suggested 'pencils down' date. Take a week to scrub code, write tests, improve documentation, etc.
Concerning their sugestion and the timeline, we are almost on schedule (on the original plan we had item #2 on the first release and item #4 on the second release*). Items #1 and #3 will certainly get done on time (before August 16). I also wish to deliver item #2 on time, which is an optional UPnP feature but is very important and widely used.

* libeupnp-av has been postponed in favour of the IGD GUI as the A/V specification is extremely extensive and the IGD GUI is a simpler showcase for the project - at least for now.

Enough with the talk, pants on and code!

Friday, August 07, 2009

Eupnp, Elementary, IGD

Nothing fancy for now, but we're on our way to end user tools :). This little application already lists and searches for Internet Gateway Devices using Eupnp and Elementary.

Next step will be about adding a dialog for controlling these devices (easy to do) but I need to strike-through one last TODO item before adding it (embedded devices parsing).

Monday, August 03, 2009


I finally managed to get XML fetching (devices and services) done. I lost a lot of time going against libcurl directly, so the solution my mentor proposed just fit: export a core download function that is provided by the integration toolkit (e.g. ecore_con_url from ecore toolkit).

I'll try to make up for the time lost and move on now for controlling!


Friday, July 31, 2009

Eupnp: last sprint

I just finished implementing the SCPD (service XML descriptor) parser, which leads me to the next phase: Control.

I'll be implementing Control in the next two weeks, which will be about using libcurl to perform HTTP POST calls with some attributes wrapped in a SOAP XML. The biggest problem about this is making libcurl work properly (I have just had two days debugging a stack corruption which lead nowhere, result is feature suspended).

Finished this task, I'll implement a simple application using elementary for listing local devices and information about them (i.e. universal control point). This is good to demonstrate functionality.

This will be followed by the control point A/V spec, task which hopefully I'll finish before GSoC pencil's down date (16 August).

Sunday, July 05, 2009

GSoC report on Eupnp

Though my release plan on the original schedule wasn't completely met on 30th June, I managed to finish some very important parts:
  • eupnp-core (code): virtual core module of the library, responsible for fd handling, timers and idlers. It's designed to make simple the integration with common event loops out there (ecore, gobject, qt). It's basically a mapping to event loop functions, see eupnp-ecore below.
  • eupnp-ecore (code): library that integrates eupnp with ecore event loop. Already contains a working example.
  • UPnP step 2, description: parsing device root XML description and services descriptions. This part is almost finished and will be followed by controlling devices (Step 3, SOAP support)
It's very likely that I'll finish SOAP support in these next two weeks. The other remaining task (event notification) will be easy to implement since most of the code is already done (HTTP message parser).

The second release will be using the library on end-user programs for controlling media devices and internet gateway devices (routers, etc). I hope I can finish this part on time and have more time for docs (which are already being written along with the code) and bug fixes.

  • dieb_ @ #edevelop
  • andre.dieb at

Thursday, June 04, 2009

Eupnp event bus and new deps

Since UPnP is event-driven itself, I have implemented a Event Bus. It is a channel onto publishers can publish events and into which subscribers can subscribe for a specific type of event (one-to-many).

The goodies about this bus is that it centralizes all eventing in one point, so, gets simpler to implement, debug, optimize and also use. The API got very simple and short, an example can be found on my example-test program.

One thing I'm still thinking about adding in this stage is multiple events subscription. I'm defining the events with independent constants (powers of 2), so it's possible to subscribe to a set of events by combining them with bitwise OR. One thing that I'm still wondering about is the frequency users would use this.

Apart from that, I have chosen two new dependencies:
  • libcurl: will be used to fetch XML's, perform soap POST's
  • libxml2: I'll use a SAX XML parser for "read-as-you-go" parsing, harder to implement but benefits in almost every way comparing to DOM.
I want to give the fetch result directly to the SAX parser, but I'm still studying the APIs to learn how to do it properly. While the XML doc is received and parsed, it'll be mounting the DeviceInfo object, without requiring a memory-expensive DOM tree.

Tuesday, May 19, 2009

Projects status

Last week I was very busy with the university and finishing up BRisa tasks, so I didn't have time to code for Eupnp, even though I couldn't take it out of my mind (ideas!).

The status is:
  • BRisa: almost ready for release (may happen sometime this week, we expect friday)
  • Eupnp: detected a 8 byte leak on inet_ntoa (well known in the community), next steps will be implementing a cache for known devices (SSDP), add filters for search targets (CP) and starting the device builder module
I'll be working on these tasks this week and on the next week I'll have to halt a bit for some university tests. I expect to have some results until friday.

Wednesday, May 06, 2009

Zaroo Leaks!

Grinding Leaks:
  • cd eupnp
  • ./
  • make
  • Create the lt binary under .lib: EINA_ERROR_LEVEL=3 ./src/bin/eupnp_basic_control_point
  • Valgrind it: EINA_ERROR_LEVEL=3 valgrind --leak-check=full --num-callers=6 --show-reachable=yes src/bin/.libs/lt-eupnp_basic_control_point
==17842== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 23 from 1)
==17842== malloc/free: in use at exit: 0 bytes in 0 blocks.
==17842== malloc/free: 36 allocs, 36 frees, 7,265 bytes allocated.
==17842== For counts of detected errors, rerun with: -v
==17842== All heap blocks were freed -- no leaks are possible.

Zaroo leaks!

Test construction failure leaks: two instances, one binds to the address, the other can't bind, check for leaks

[eupnp_udp_transport.c:85] eupnp_udp_transport_prepare() Error binding. Address already in use
[eupnp_udp_transport.c:135] eupnp_udp_transport_new() Could not prepare socket.
[eupnp_ssdp.c:324] eupnp_ssdp_server_new() Could not create SSDP server instance.
[eupnp_control_point.c:86] eupnp_control_point_new() Could not create control point.
==18183== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 23 from 1)
==18183== malloc/free: in use at exit: 0 bytes in 0 blocks.
==18183== malloc/free: 23 allocs, 23 frees, 1,490 bytes allocated.
==18183== For counts of detected errors, rerun with: -v
==18183== All heap blocks were freed -- no leaks are possible.

Monday, May 04, 2009

GSoC Eupnp Progress

  • Repository:
  • Cloning: git clone git://
  • Testing: ./, make, make install, try the binaries under src/bin with EINA_ERROR_LEVEL=3 if you want complete debug.
These are the features I already pushed (see update below):
  • UDP transport component;
  • SSDP HTTP basic request parser;
  • SSDP Discovery Request (M-Search);
  • Control Point and SSDP classes;
  • Working basic control point for testing features described above

Update: Gustavo Barbieri (project mentor) helped me a lot improving everything and also fix the repository tree that had a 'trunk' folder. So I created a new tree and pushed the files already improved.

From now on, only incremental commits! :-)

Monday, April 27, 2009

GSoC 2009: let the games begin

For this past week I've been learning autotools as fast as I could, and as pragmatic as I could. I don't want to lose much time with this. I also managed to finish a lot of pendencies, so I'll have more time to write early code - before May 23 (official start-coding date).

Monday, April 20, 2009

GSoc 2009: accepted!

I'm very happy. I've been accepted as student for Google Summer of Code 2009. My project is associated with The Enlightenment Project organization and it will be implementing a UPnP control point library in C. A brief description of the project can be found here.

I have already started coding a bit and done some bits on the SSDP server (finished the multicast listener).

The code will be available on a git repository on github. You can access the tree here and I'll be pushing the stuff soon.

Wish me luck!