[Jingle] [Fwd: [Summit] Jingle topics (was: Re: introduction)]

[Peter Saint-Andre]

Forwarding to jingle (AT) xmpp (DOT) org for further discussion if necessary. I
don't think we covered all of these items at the summit...

/psa

-------- Original Message --------
Date: Sun, 20 Jul 2008 17:27:45 -0700
From: Peter Saint-Andre <stpeter (AT) stpeter (DOT) im>
To: XMPP Summit <summit (AT) xmpp (DOT) org>
Subject: [Summit] Jingle topics (was: Re: introduction)

Robert McQueen wrote:
> Hi everyone,
>
> I'm one of the directors of Collabora (http://www.collabora.co.uk/), we
> do development and consultancy on open source real-time communications
> and streaming stuff. I'm the lead developer of the Telepathy real-time
> communications framework (http://telepathy.freedesktop.org/), which is
> used on the Nokia internet tablets, and the OLPC project. XMPP is our
> preferred supported protocol, and we've worked a lot on Jingle (our
> support, and the specs) over the past year or two.

Hi Rob, I'm excited that you'll be there!

> We're currently updating our implementation to the current state of
> play, and would like to meet with fellow Jinglers to nail down a whole
> load of outstanding Jingle issues:

Thanks for the list of open issues. Let's get this stuff worked out so
that we can advance these specs to Draft already. :)

> * getting rid of weird SIP hang-overs in jingle like profiles

+1

> * sensible fallbacks between jingle transports, content-replace is kinda
> loss

I think we also need to figure out renegotiation of an existing session
(IMHO this is closely connected to what you are calling "sensible
fallbacks").

> * TLS over jingle stuff

By this do you mean (1) encryption of Jingle-negotiated media such as
voice or video (e.g., via SRTP or DTLS) or (2) use of Jingle to
negotiate an encrypted end to end XMPP session as in XEP-0247?

> * jingle descriptions for "arbitrary stream protocol" transportation,
> like any DNS-SD service (rfb, chess, whatever)

Looking forward to hearing what this is.

> * other APIs which are useful for collaborative applications (shared
> document models, but not weird bonghits like SXE)

Please, tell us what you really think. :) Is this connected to the stuff
we talked about in Brussels for exchanging DOM object changes (or even
random JSON data etc.)?

> * stacking/layering/nesting jingle transports, so we can say "here's a
> reliability layer, on top of an unreliable transport" or "here's a TLS
> layer on top of this unsecure transport"

Again, I'm looking forward to hearing your descriptions of these.

> I'm also interested in seeing if people have interest in Jingle
> descriptions for including existing stream protocols, datagram
> protocols, and D-Bus IPC, so that we can standardise/interoperate
> Telepathy's Tubes (http://telepathy.freedesktop.org/wiki/Tubes,
> http://people.collabora.co.uk/~robot101/telepathy-guadec-2008.pdf).

OK.

> I'll be arriving a little late on Monday morning because I'm speaking on
> Sunday afternoon at LUG Radio Live in the UK, but will stick around
> until Thursday so I can put in an appearance at the Oregon Brewers
> Festival (http://www.oregonbrewfest.com/). :D

Good plan. I didn't have the foresight to work that in, so I'm leaving
earlier in the day on Thursday. :/

> Look forward to seeing you there.

Indeed!

/psa

[Robert McQueen]

Peter Saint-Andre wrote:
> Robert McQueen wrote:
>> * getting rid of weird SIP hang-overs in jingle like profiles
>
> +1

Lets just do that. Redundant stuff is bad. We can determine the
transport and that RTP is beong used by inspecting the XML namespaces in
use, and determine whether SRTP is there by looking for the appropriate
element. This is a good reason to put the SRTP element in the RTP
description XEP itself, because otherwise a gateway could fail by not
understanding the <crypto xmlns="...srtp"> element, and turning the SRTP
offer into a plain RTP/AVP one.

>> * sensible fallbacks between jingle transports, content-replace is kinda
>> loss
>
> I think we also need to figure out renegotiation of an existing session
> (IMHO this is closely connected to what you are calling "sensible
> fallbacks").

I think we covered this, transport-replace seems OK, and we're going to
try and avoid other renegotiations if we can get away with it. :)

>> * TLS over jingle stuff
>
> By this do you mean (1) encryption of Jingle-negotiated media such as
> voice or video (e.g., via SRTP or DTLS) or (2) use of Jingle to
> negotiate an encrypted end to end XMPP session as in XEP-0247?

I meant TLS'd XMPP streams over Jingle. Seems pretty good, most of what
we need to do is just define some best practices for clients to generate
keys, and do some continuity checking at least.

>> * jingle descriptions for "arbitrary stream protocol" transportation,
>> like any DNS-SD service (rfb, chess, whatever)
>
> Looking forward to hearing what this is.

I think I should just write this, it's pretty trivial, and we really
want it for Telepathy Tubes. Just so you can invite your contacts to
connect to your local DAAP/VNC/HTTP/whatever service.

>> * other APIs which are useful for collaborative applications (shared
>> document models, but not weird bonghits like SXE)
>
> Please, tell us what you really think. :) Is this connected to the stuff
> we talked about in Brussels for exchanging DOM object changes (or even
> random JSON data etc.)?

Yeah, we didn't talk at all about this. I think SXE is pretty crazy
because it seems like some huge explosion of the XML to make totally
minor changes. If you have a shared DOM, you can use existing standards
like xpath to name an existing node and say "insert a child node",
"delete this node", or "replace this node". I didn't look at SXE in a
lot of detail, but it seemed like an incredibly verbose way of achieving
not a lot more than this. I don't think it actually addresses the hard
stuff like how do you keep a consistent document model across multiple
clients, which is what actually makes collaborative applications hard to
develop, not the "how do I stuff bytes from A to B".

The background to this is that OLPC has fed back to us and said that
they don't think Telepathy Tubes are a particularly good API for writing
collaborative applications on. I don't disagree, we just had two
requirements in mind - one was allowing people to re-use existing wire
protocols when appropriate, but have Telepathy take care of NAT punching
or relating, which we invented stream tubes for, and the other was
exposing a multi-user primitive that we could implement on top of MUC or
multicast, which we invented D-Bus tubes for.

We were hoping to get some push back from OLPC activity authors about
what they'd rather see in the APIs, because we had very unclear
requirements when we started out, other than "enable collaborative
applications somehow", but we never really got that feedback. People who
try to write collaborative activities on OLPC in general fail to do it,
because they still have to know distributed computing and network stuff,
even for simple things like a game which has shared state and only one
player changing it at a time.

Lots of people (Inkscape, Abiword) have also got code which just speaks
XMPP directly with their own ad-hoc extensions, but this doesn't have
any prospects of interoperating, sends lots of data through IBB, doesn't
encourage code sharing (Telepathy's NAT traversal or Jingle
implementation isn't really available, nor UI elements, etc), and
wouldn't work in situations where Telepathy could do something different
(like our link-local backend's multicast MUCs).

I'd like to see something that will a) allow us to make life easier for
application developers, b) be implementable by us on both link-local and
server-based XMPP (and maybe even other protocols), and c) ideally be
interoperable with other clients.

This probably isn't the right forum to discuss this however, but feel
free to forward me around a bit more... :D

>> * stacking/layering/nesting jingle transports, so we can say "here's a
>> reliability layer, on top of an unreliable transport" or "here's a TLS
>> layer on top of this unsecure transport"
>
> Again, I'm looking forward to hearing your descriptions of these.

We kinda decided a generic stacking thing was a bad idea, but I did come
up with a way to run several reliable streams on top of one pseudo
TCP-on-ICE connection, which I'll come back to in my later post on
pseudo TCP.

>> I'm also interested in seeing if people have interest in Jingle
>> descriptions for including existing stream protocols, datagram
>> protocols, and D-Bus IPC, so that we can standardise/interoperate
>> Telepathy's Tubes (http://telepathy.freedesktop.org/wiki/Tubes,
>> http://people.collabora.co.uk/~robot101/telepathy-guadec-2008.pdf).
>
> OK.

This is the same as what I meant above. :D

> /psa

Regards,
Rob

[Peter Saint-Andre]

Robert McQueen wrote:
> Peter Saint-Andre wrote:
>> Robert McQueen wrote:
>>> * getting rid of weird SIP hang-overs in jingle like profiles
>> +1
>
> Lets just do that. Redundant stuff is bad.

Done in my working version of XEP-0167.

> We can determine the
> transport and that RTP is beong used by inspecting the XML namespaces in
> use, and determine whether SRTP is there by looking for the appropriate
> element. This is a good reason to put the SRTP element in the RTP
> description XEP itself, because otherwise a gateway could fail by not
> understanding the <crypto xmlns="...srtp"> element, and turning the SRTP
> offer into a plain RTP/AVP one.
>
>>> * sensible fallbacks between jingle transports, content-replace is kinda
>>> loss
>> I think we also need to figure out renegotiation of an existing session
>> (IMHO this is closely connected to what you are calling "sensible
>> fallbacks").
>
> I think we covered this, transport-replace seems OK, and we're going to
> try and avoid other renegotiations if we can get away with it. :)

Yep, +1.

>>> * TLS over jingle stuff
>> By this do you mean (1) encryption of Jingle-negotiated media such as
>> voice or video (e.g., via SRTP or DTLS) or (2) use of Jingle to
>> negotiate an encrypted end to end XMPP session as in XEP-0247?
>
> I meant TLS'd XMPP streams over Jingle. Seems pretty good, most of what
> we need to do is just define some best practices for clients to generate
> keys, and do some continuity checking at least.

Right. Separate thread, separate list. :)

>>> * jingle descriptions for "arbitrary stream protocol" transportation,
>>> like any DNS-SD service (rfb, chess, whatever)
>> Looking forward to hearing what this is.
>
> I think I should just write this, it's pretty trivial, and we really
> want it for Telepathy Tubes. Just so you can invite your contacts to
> connect to your local DAAP/VNC/HTTP/whatever service.

Whee, tubes! Sounds like fun!

>>> * other APIs which are useful for collaborative applications (shared
>>> document models, but not weird bonghits like SXE)
>> Please, tell us what you really think. :) Is this connected to the stuff
>> we talked about in Brussels for exchanging DOM object changes (or even
>> random JSON data etc.)?
>
> Yeah, we didn't talk at all about this. I think SXE is pretty crazy
> because it seems like some huge explosion of the XML to make totally
> minor changes. If you have a shared DOM, you can use existing standards
> like xpath to name an existing node and say "insert a child node",
> "delete this node", or "replace this node". I didn't look at SXE in a
> lot of detail, but it seemed like an incredibly verbose way of achieving
> not a lot more than this. I don't think it actually addresses the hard
> stuff like how do you keep a consistent document model across multiple
> clients, which is what actually makes collaborative applications hard to
> develop, not the "how do I stuff bytes from A to B".
>
> The background to this is that OLPC has fed back to us and said that
> they don't think Telepathy Tubes are a particularly good API for writing
> collaborative applications on. I don't disagree, we just had two
> requirements in mind - one was allowing people to re-use existing wire
> protocols when appropriate, but have Telepathy take care of NAT punching
> or relating, which we invented stream tubes for, and the other was
> exposing a multi-user primitive that we could implement on top of MUC or
> multicast, which we invented D-Bus tubes for.
>
> We were hoping to get some push back from OLPC activity authors about
> what they'd rather see in the APIs, because we had very unclear
> requirements when we started out, other than "enable collaborative
> applications somehow", but we never really got that feedback. People who
> try to write collaborative activities on OLPC in general fail to do it,
> because they still have to know distributed computing and network stuff,
> even for simple things like a game which has shared state and only one
> player changing it at a time.
>
> Lots of people (Inkscape, Abiword) have also got code which just speaks
> XMPP directly with their own ad-hoc extensions, but this doesn't have
> any prospects of interoperating, sends lots of data through IBB, doesn't
> encourage code sharing (Telepathy's NAT traversal or Jingle
> implementation isn't really available, nor UI elements, etc), and
> wouldn't work in situations where Telepathy could do something different
> (like our link-local backend's multicast MUCs).
>
> I'd like to see something that will a) allow us to make life easier for
> application developers, b) be implementable by us on both link-local and
> server-based XMPP (and maybe even other protocols), and c) ideally be
> interoperable with other clients.
>
> This probably isn't the right forum to discuss this however, but feel
> free to forward me around a bit more... :D

Yeah the right list is probably here:

http://mail.jabber.org/mailman/admin/sharedediting

Or something. Maybe we need a "collaboration" list.

>>> * stacking/layering/nesting jingle transports, so we can say "here's a
>>> reliability layer, on top of an unreliable transport" or "here's a TLS
>>> layer on top of this unsecure transport"
>> Again, I'm looking forward to hearing your descriptions of these.
>
> We kinda decided a generic stacking thing was a bad idea, but I did come
> up with a way to run several reliable streams on top of one pseudo
> TCP-on-ICE connection, which I'll come back to in my later post on
> pseudo TCP.

/me nods

>>> I'm also interested in seeing if people have interest in Jingle
>>> descriptions for including existing stream protocols, datagram
>>> protocols, and D-Bus IPC, so that we can standardise/interoperate
>>> Telepathy's Tubes (http://telepathy.freedesktop.org/wiki/Tubes,
>>> http://people.collabora.co.uk/~robot101/telepathy-guadec-2008.pdf).
>> OK.
>
> This is the same as what I meant above. :D

OK, cool.

/psa

[Dirk Meyer]

Hi,

Robert McQueen wrote:
> Peter Saint-Andre wrote:
>> Robert McQueen wrote:
>>> I'm also interested in seeing if people have interest in Jingle
>>> descriptions for including existing stream protocols, datagram
>>> protocols, and D-Bus IPC, so that we can standardise/interoperate
>>> Telepathy's Tubes (http://telepathy.freedesktop.org/wiki/Tubes,
>>> http://people.collabora.co.uk/~robot101/telepathy-guadec-2008.pdf).
>>
>> OK.
>
> This is the same as what I meant above. :D

Let me understand it right. Simply speaking Telepathy's Tubes are a
way to connect local Dbus applications to other machines in the LAN
and even on the Internet. This idea his very similar to something I'm
working on except that I don't plan to use Dbus for application
specific communication and want to use XEPs for that, too. We really
should work together on how to let applications interact with
eachother on the internet.


Regards,


Dirk

--
Christmas is weird. It's the only time of year when we love to sit in
front of a dead tree and eat candy out of your socks.

[Dirk Meyer]

Peter Saint-Andre wrote:
> Robert McQueen wrote:
>> I meant TLS'd XMPP streams over Jingle. Seems pretty good, most of what
>> we need to do is just define some best practices for clients to generate
>> keys, and do some continuity checking at least.
>
> Right. Separate thread, separate list. :)

What thread? I'm reading devel, social, jingle and standards-jig using
gmane. Do I need another list or are you talking about OAuth (which is
nice but IMHO something different).


Regards

Dirk

--
Whenever a system becomes completely defined, some damn fool discovers
something which either abolishes the system or expands it beyond recognition.

[Peter Saint-Andre]

Dirk Meyer wrote:
> Peter Saint-Andre wrote:
>> Robert McQueen wrote:
>>> I meant TLS'd XMPP streams over Jingle. Seems pretty good, most of what
>>> we need to do is just define some best practices for clients to generate
>>> keys, and do some continuity checking at least.
>> Right. Separate thread, separate list. :)
>
> What thread? I'm reading devel, social, jingle and standards-jig using
> gmane. Do I need another list or are you talking about OAuth (which is
> nice but IMHO something different).

Oh sorry, the thread hasn't started yet. :) I have too many actions
items today, and I haven't gotten to that one yet...

/psa

[Robert McQueen]

Dirk Meyer wrote:
> Let me understand it right. Simply speaking Telepathy's Tubes are a
> way to connect local Dbus applications to other machines in the LAN
> and even on the Internet. This idea his very similar to something I'm
> working on except that I don't plan to use Dbus for application
> specific communication and want to use XEPs for that, too. We really
> should work together on how to let applications interact with
> eachother on the internet.

Yes, that's right. My current thought is that we should have a Jingle
XEP content description which says "this stream protocol is in use", and
refer to it by its DNS-SD service name. We can also define a mapping
between the DNS-SD extra parameters and the Jingle offer.

We'd then be able to signal a Jingle session to have our streams sent
over a raw TCP transport, SOCKS5 or IBB, or when we have code/XEPs for
it we can also make use of a reliability layer on top of ICE-UDP. We can
say that the offerer is the listener, and the responder can establish as
many connections over the stream as he wishes while the session is open.
This will allow protocols like HTTP or DAAP where the client can connect
multiple times to the server to be used.

Also we could define something to include in XEP-0115 entity
capabilities to list the DNS-SD protocols you're able to take part in,
such as:
<feature var="jingle:yada:stream#vnc"/>
<feature var="jingle:yada:stream#http"/>

> Regards,
>
> Dirk

Regards,
Rob