#6: Christopher Webber - Media Goblin
13 Oct 2013
Christopher Webber talks about Media Goblin, a decentralized media publishing platform. How can we have a federated alternative to services like Flickr, YouTube and SoundCloud?
Kindly transcribed by: David Hansen
Irina Bolychevsky: Hi everybody! Today we’re talking to Christopher Webber, the founder and lead developer of MediaGoblin, a free-software media publishing platform that anybody can run. Chris, thank you so much for joining us. Why don’t you introduce yourself and tell us a little bit more about MediaGoblin?
Christopher Webber: Sure. So I am Chris Webber. I am the lead developer. I’ve been working on free and open-source software things for some time. I used to work at Creative Commons not too long ago, and before that at the Participatory Culture Foundation. So I’ve been doing these things for a while. But I started MediaGoblin — well, co-started MediaGoblin — with a number of people a number of years ago.
So, what MediaGoblin is, I mean, you already said it’s a media publishing system for the web, but if you wanted to try to think of what it’s kind of analogous to, you could think of it as a replacement for Flickr and YouTube and stuff like that. So you can have video and images and stuff all in the same place.
Irina: So it’s essentially open-source versions of these popular services but kind of targeted, so you’re not centrally hosted; and the services, they’re primarily for running on your own server?
Christopher: Right. So you can — anybody can — run one of these on their own server. I mean, in that sense we already are at a very significant amount of decentralization. We’re actually doing more work right now to actually make that even better, which, I don’t know if I should just go into rambling about that or if I should do that later. But yes, that’s the gist.
Irina: Ok. But I think before we get into that, so, what makes MediaGoblin unique? I mean, it’s a whole set of things. Why don’t you tell us about what it actually lets you do, and what kind of things it covers?
Christopher: Ok. So if you think about media publishing as a gallery, I mean, I started this because I’m a hybrid programmer and artist and I wanted a place where I could put a bunch of my own works, myself, all in one place. So you know, the closest thing that comes to doing this sort of thing that has multiple types of media is — DeviantArt does this in some ways, Flickr kind of does this with both having some image support and some kind of crappy video support. And so, in a certain sense it is useful as an artist gallery, an artist portfolio-type tool.
So that’s one thing that I think sets it apart, is that we do have support for not just images, video, but you can actually plug in a whole variety of media types and have them all in one gallery. Like you can add document support, video support, adding documents and converting them to PDFs, and still showing them kind of in a nice unified way. We have 3D model support, so if you wanted to build a replacement for something like Thingiverse for a gallery of objects you might 3D print, you might have that.
But I mean, the reasons why somebody might want such a thing is, you know, the general theme of Redecentralize.org is trying to give power back to people and trying to get — to take things out of these gigantic mega-sites. And so of course we — that’s one of the other goals as well.
Irina: Ok. So as a user — I think if we step a bit — so who do you think your intended key audience or user group is?
Christopher: Ok. So our key audience is presently — it’s largely people — the reality-present audience is people who are very interested in these issues from some kind of idealogical concern. We are trying to build the user interface, though, in a way so that that’s not the case. But the present set of people who are interested are mostly artists who have an intersection with these types of issues. But our general 1.0 goal is to get to a place where we have all the features in place, where pretty much anyone is interested in jumping in, regardless of whether they’re a free and open-source software nerd or not.
Irina: Sure. So basically as somebody who has videos or photographs, this allows them to essentially have a showcase or a site online which displays all this kind of media, but it’s hosted yourself, somewhere yourself, right? Do you provide a way for people, if they don’t have their own server, to host it somewhere?
Christopher: Well, you don’t have to run your own server necessarily, you could connect to somebody else’s server.
Irina: Ok, and how can people do that?
Christopher: Well, there’s a list of publicly available sites. On the site, if you go to, it has a ‘Use It’, ‘Join Us’, ‘Run It’ thing, and you could click the ‘Use It’ and that will bring you to a list of that. And so there are some publicly available sites at the moment that you could just connect to and start using. But, of course, when you’re. . .
Irina: But ideally, you know, it’s a chance for you to own your own data and control where it is. So if you were to tell me like what is your motivation, in the sense of why does it make a difference? Why, as a user — from my perspective — why is it in my interest to download MediaGoblin as opposed to using a site such as Flickr or YouTube where there’s a big community and there’s a set of people who are already looking at those sites?
Christopher: Well, I mean, there’s certainly a lot of convenience to using sites like Flickr and YouTube and stuff like that. The reasons why I think as a user you might want to do so is, I think there is also a certain amount of fragility that comes to using sites like those. You have no guarantee that the site that you’re depending on, in terms of Flickr or YouTube or something like that, is going to persist into the future. GeoCities was the hottest thing back in, you know, 1998, and that no longer exists. And if, as an artist, these things are really important to you, you never know when something like that might end up going away.
There were a lot of people who were upset very recently about Google Reader going away. And so there’s no — a service will continue to exist as long as it’s within a corporation’s interest to do so. And if everybody’s videos are all on YouTube and then eventually YouTube goes away, then a whole lot of videos from the Internet basically all disappear. And that would kind of suck.
Irina: Yeah, then where would you go for all these cat videos? But to sort of step back, I mean obviously these sites shut down, but you can generally export your data out and put it somewhere else, so. . .
Christopher: Well, you kind of can. I mean, it’s usually a big pain in the butt to actually do so. I’m actually not sure if there’s a nice way to export your data on YouTube presently, though. Maybe there is.
Irina: I don’t know either, actually.
Christopher: I mean, it’s partly also that the issue is that this type of stuff is becoming systemic, though. To be bluntly honest — and this is something I think that we often don’t talk about these things — is that going the route of using something like YouTube and Flickr and stuff like that is at the moment the easiest thing to do, often. Because it’s just, it’s there, and it’s set up, and etc. I would like there to actually be options so that that’s the same case with MediaGoblin. Where there are some sites that are fairly well supported and you can just jump on them, but they’re not the only MediaGoblin sites, right?
And that doesn’t quite exist in terms of something like, with a very dedicated amount of MediaGoblin hosting, and I think the ideal future is to actually have kind of a combination of those two things — encourage as many people as possible to run things for their family and friends, and then for the people who can’t, have a few companies that are basically running things, but it’s not just one or two. It’s like a larger set of them. Like the way that, you know, there’s wordpress.org and wordpress.com, but there are a ton of WordPress site installs out there, right? It’s not just WordPress the company installing WordPress. So that’s the ideal future I think.
Irina: And what do you think it would mean if MediaGoblin would see mass uptake?
Christopher: Well, I think that — in terms of what it would mean — well, the obvious answer is it would mean moving away from a lot of the problems that I’ve said. I actually do think that there are a number of things that we need to complete in order to get there. I mean, the ideal thing is that we eventually get to the point where MediaGoblin uptake is so easy for people to do that they don’t really have to think that much about it. We’re definitely not at that state right now.
I mean, the biggest answer that I can give as to what it would mean would basically be reversing a lot of the problems that I’ve been talking about — you know, the fragility of the centralization. Also, we would like to move towards — one of the things that’s on our road map is issues of privacy and stuff like that. And one thing that’s not very well addressed is actual privacy when it comes to sharing of media. So you want to put up some images that are just for your family and friends, and very much so actually just for your family and friends. Not just your family and friends and this large corporation that might do whatever, you know?
And that’s also a concern for political activists and stuff like that. So I think, you know, the world in which people don’t have all their data — which includes media, the type of media that’s encompassed by MediaGoblin — being run through just huge pipes; it’s a world then which would return a lot of more autonomy to users in that type of way.
Irina: Yeah, exactly. And I guess there’s also the sense that — even, of course, services like Flickr or Picasa or the Google equivalent let you have your private albums. Obviously there are ways and the GCHQ and the NSA can always go and find these images if they wish.
Christopher: And do!
Irina: And do. And presumably others can as well. So have you thought about security or encryption, and is that something that is sort of built in or on the roadmap? Firewalls?
Christopher: Yeah, so, in order to get to the — so I think there are actually a couple of, like, levels to be able to get there, first of all. The first things is, is that presently MediaGoblin instances don’t talk to each other. And this is one of the goals for 1.0. Right, so in order to really get to the point where you’re like — oh, so sure, actually at present, already, if you have MediaGoblin instances for your political activist group or your family and friends, it’s already the case that you could actually — although we actually don’t have a lot of privacy, like private sharing stuff — I mean you could basically put it behind something so that there is a certain amount of protection there.
But the real thing that we need, in order to even start thinking about this type of stuff, is — how do we even connect MediaGoblin sites together. Right? Because if you just have a bunch of separate people having their own separate MediaGoblin sites, but then they’re all on their tiny little islands and can’t really leave those islands, that’s not actually great at all, right? That’s not really an interesting thing.
So that’s why federation is so important. And that’s actually one of the big things that we’re working on presently is federation support in MediaGoblin. And federation means basically hooking together two sites with the same kind of — even though they’re separate sites — they’re kind of connected together in the way that, like email servers, although entirely separate, you have the same kind of cohesiveness as if they were on the same server. So federation is one of the big things that we’re working on at the moment.
Irina: So I think it would be useful to explain that a little bit more. What would federation actually practically mean?
Christopher: Ok, so there’s two ways to describe that. I can describe that on the nerdy technical level and I can describe that on the user level where you should never have to know about that, right? So. . .
Irina: Both would be awesome!
Christopher: Ok, alright. So let me go into the technical level first, and possibly bore people who are not interested in that before I get to the real level. So I guess, whatever! But the technical level is — we’re actually starting work on this. We had an awesome Outreach Program for Women intern this summer who did a bunch of work on this. Her name is Jessica Tallon. She is super great!
We are implementing an API that’s called the pump API. It’s used by pump.io, which is kind of the successor to StatusNet slash — which is what identi.ca ran, and we’re using that protocol. We’re still in the process of getting that implemented. There have been other things going on this summer also to try to make it so that it’s easier for other developers of other projects to get the same protocol integrated, because there’s a lot of overlap in terms of what federation means between a whole bunch of different types of services, not just media sharing services.
So as opposed to trying to get people to re-implement that wheel constantly, we’ve also been — Jessica has been — working on a library called PyPump that should make this easier for Python developers. So that’s kind of one of the things that’s going on in progress. Well, in progress — that’s going on to try to advance this. But that’s a very nerdy, technical-level side of things. You can read the pump API; just do a search for ‘pump API’ to pull it up. And you can check out pump.io if you actually want to take a look at a service that is doing federation and is using the pump API right now. It’s kind of like a much more minimalist version of Twitter/Facebook right now that can have multiple servers talking to each other.
So what would it mean if we actually had federation, right? And that’s all actually taken care of and users don’t have to see it? Obviously we don’t want users to actually think about ‘federation is happening in the background’. So the ideal type — so even already if you put a video up on a MediaGoblin site, and say you just want to make it public and you just want people to see it. That’s already possible. You can, you know, for some users, they don’t really care whether it’s on Flickr, or whether it’s on Vimeo or YouTube or a MediaGoblin site, they just want to watch the video. And that already exists.
But what if you want to have people comment on it between instances? And what if you want to have people add it to galleries? Like, you know, for example on Flickr you have Flickr pools which allows you to — sorry, I’m rambling.
Irina: So, no! I just wanted to clarify — which is, you mentioned commenting between instances. What do you mean by that?
Christopher: Oh, ok. So say you have a user on my MediaGoblin instance. So, ok. Say you and I are on separate MediaGoblin instances, right?
Christopher: You’re on the ‘Goblin Party’ site and I’m on a My Little Pony fansite, ok? So — I’m not actually a My Little Pony fan, though I am massively amused by that culture of. . .
Irina: Of course not! Of course not.
Christopher: But anyway, so what would happen is, is that you could have a — your user wants to be able to comment on my video. And how do you actually do that? You have to be able to sign in with your user on your account on the site that I’m on.
Irina: Does MediaGoblin come with user accounts and user. . .
Christopher: Yeah. So you can have one MediaGoblin site that has a bunch of users on it. And actually, we just. . .
Irina: What do you. . . ?
Christopher: We just have a new. . .
Irina: What do you. . . ?
Irina: Sorry! We just did that twice. I was just going to say — do you kind of plug in some other commenting functionality or is that all built in to MediaGoblin?
Christopher: It’s built in to MediaGoblin, yeah. And we actually just pushed out a new release that has multiple media type plug-ins. We had another intern this summer who — named Rodney Ewing, and he did an awesome job. We now have OpenID and Mozilla Persona support, in addition to basic login, username/password on your own site.
Irina: Wait, OpenID is getting shut down, isn’t it?
Christopher: Oh, so actually this is an interesting topical discussion, right? So myopenid.net is shutting down, but OpenID the standard is not shutting down. So one of the big flagship instances of OpenID is going away, but the standard of OpenID is not going away. So if you’re using an OpenID thing on some other service, it’ll still exist. This is a little bit like if Gmail shut down. Would email go away if Gmail shut down? No, email would still exist; email existed before Gmail existed. But a whole lot of email accounts at once would go away. That’s the difference. So OpenID is actually already a protocol that’s built to try to make these things be done right.
Irina: But anything that’s hosted on the one that’s shutting down, that will disappear. This is where federation comes in, which is where you want to make sure that there are copies between different services.
Christopher: Right. You want to be encouraging people to not be on these huge mega-sites, basically. And so, you know, I’m saying that the ideal future involves both a bunch of smaller sites and also some larger sites run by companies that are — but you don’t want it to be that like, oh, ‘mediagobliner.er’ eventually shuts down and everybody’s like, ‘Oh no! MediaGoblin shut down!’ Right? You want there to be enough MediaGoblin sites where people don’t think that a whole section of the Internet shut down just because one site shut down.
So that’s one of the problems we’re having right now, is even when you end up — or do — have a federated protocol, the temptation that people have is to just use the site that’s already there and works. And there’s this kind of problem, where, in a certain sense it helps federated protocols gain adoption when there’s this mega-site that does it. But on the other hand that makes that much more fragile, because what if eventually, you know Google was originally implementing a chat protocol called XMPP, or Jabber, that was federated. Eventually Google decided now they’re going to — well Google indicated that they may be shutting that down and they’re probably moving to Hangouts only. And so that means that actually this mega — it was convenient for XMPP and Jabber that you can easily — anybody who had a Gmail account you could end up talking to, even if you weren’t using Gmail.
So you could end up discussing — you could end up using their IM with that person, right? So I have my email@example.com account; anybody with a Gmail account I can still IM. But pretty soon Gmail is going to — theoretically, it looks like Google is going to shut that down, and that means that since we got so reliant on Google conveniently providing this for us, it means that our federated protocol was not as strong. So there is this element that, even if we get federation, we really have to, as much as possible, encourage people to not just be kind of lazy and jump into these mega-sites.
So like, I’m adding complexity to this conversation, I know! And I’m actually, I’m not always the best PR person. I’m like the lead developer who thinks about all the issues. So the ideal PR person would be like, ‘Yeah, we’re just going to scale things up and add the federation, and this’ll be super simple. MediaGoblin’s great! It’s the future.’ The problem with interviewing me is that I’m like, ‘Here are all the problems. These are the problems. We’re trying to build the solutions, and here are the problems.’ Which makes me not a really great PR person. But I’m trying to be honest. Like these are the actual problems that we have to address in order to make this stuff work. And it’s hard. We have to keep working at it.
Sorry! I know I have the Chris Webber mode of like nonstop rambling. And I’m doing it again! I just stopped you! This is what happens.
Irina: Ok. I was going to ask something, actually, but it just — that last impassioned speech kind of just stopped me dead! But, yeah, just to kind of — because I think there are lots of really interesting things about that. One of which is, like, technologically, what are the important things to maintain that balance that you were talking about between having these easy-to-go-to central places where you think, ‘Ok, that’s the instance that I should use,’ and the smaller instances.
And also, I guess — I think it’s still a little bit not clear (I might be asking a stupid question), which is when you talk about federation, and you talk about the federation of the actual data or of the protocol or of something else, because I feel like there are different strands, you know, coming in to play.
Christopher: Ok. So that’s actually something people have asked before. If you’re having two MediaGoblin sites talking to each other, are they actually cloning each other’s images and videos? Or is it just the metadata about it, right? Like the title, the subject, here’s where the thing actually lives. So the default in MediaGoblin is going to be that it’s actually — it’s going to be more like blogging, right? So blogging is where you have a lot of links to other people’s sites, but you generally don’t end up copying in the entire content. You might copy in the content, but you’re not necessarily; you’re probably linking to various people’s stuff.
So MediaGoblin will definitely be aggregating other people’s thumbnails, but you’ll probably actually — for the most part the data will actually be on that other person’s site. And the reason for that is otherwise else we might as well name MediaGoblin ‘DiscGobbler’ instead of MediaGoblin. You know, like we might have a DiscGobbler plug-in that actually allows an easy cloning of other people’s stuff, but the real goal here is to actually make it so that if one site goes down, it doesn’t take the entire ‘Internet’ with it. It takes down that segment of things. It doesn’t take down the whole thing.
But the links between them is the main thing that we’re actually federating, and the social communication, like aggregating whether or not somebody — how much somebody — liked something, being able to have this collaborative aspect and things like that.
Irina: Right. So it’s more around, sort of — because the accounts are standard, so if someone uses their account, that obviously is storing the data for the comments or likes, whatever, on there so if that goes down then that information isn’t lost in the system?
Christopher: Right. So, the other. . .
Irina: I’m just sort of checking whether that’s correct.
Christopher: Well, the other person’s site will actually probably be — if you’re actually commenting on somebody else’s site, it should be keeping an aggregate of your comments on that other person’s site and stuff like that.
So yes and no. Like, I can get into details a bit stronger, but the main difference is, the social level of things will be aggregated, but you won’t necessarily be copying everybody’s videos from the entire Internet onto your server because then your server would go down.
Irina: Sure, sure. I think I’m just — I’m trying to get back to the benefits to the user.
Christopher: Ok, the benefits to the user. I started to go into like thinking about the data structures of things. Again, I’m not always the best person because I’m like Tech Guy. So the benefit to the user is that you could have your own site. I mean, there are other benefits to federation besides just the resiliency aspect, too. There’s also the fact that — and you know, something going down isn’t always even just like it’s just going down permanently. It’s also it might be going down temporarily.
The other side of things too is also that you might want to theme your site to look a lot more personal. A company might want to run a MediaGoblin instance and might actually want to make it fairly themed, but they can still work with the rest of the web and have some kind of personal customizations there.
And the federation side of things isn’t that much more interesting except for realizing that, oh, wait a minute, I really liked Google Reader, and then Google Reader went away, and now I don’t have Google Reader anymore. And I really liked this type of thing, and then it went away. And that’s really shitty. But I guess there is a big feature, which — we started to talk about privacy and then we went away from that because I started talking about federation.
And privacy is one of the places where this actually does come into play, because you can actually have a site where it’s just your site and my site actually talking to each other. And assuming that we both have a fair amount of control over the servers that we’re actually handling between these two types of things, and we’re actually passing information back and forth, and assuming we’re actually using SSL and it’s encrypted and people are not intercepting the certificates and stuff like that, we’ve made it very difficult to actually just break in and sabotage that kind of communication that we’re actually having.
But anyway, I’ve probably given away, through this interview, why we have Deb Nicholson handle the community management and outreach type of things, because I’m like, ‘This is the things that are there,’ and ‘This is why this is important,’ and then I go on to these. . .
Irina: No, this is great! I think it’s sort of interesting, especially since I guess you’re not directly trying to, you know, have some kind of alternative or replacement for the way the Internet is at the moment, but — and correct me if I’m wrong — but to some extent a lot of the use of the Internet is people, whatever, going onto YouTube or using email. And actually if these services were hosted on individual services that talked to each other, then there wouldn’t be this problem that everything was going through one area. But is that something that you’re going to be addressing more directly in terms of actually having some sort of alternative way to communicate between people?
Christopher: I’m not sure whether you mean, like, on a user interface level, or in terms of the features of the program, or if you just mean the structure of, the way that connecting these things together are.
Irina: So the structure of how these things can connect and. . .
Christopher: Ok. So I think an interesting way to put this is, in terms of a structure connecting things, is an interesting counterpoint of trying to change the Internet to this more decentralized thing — is actually, in a certain sense, we’re trying to take the Internet back. Like, the Internet was designed with these ideals of a super resilient network. And the web was designed so that you could have a whole lot of different servers, and it’s not controlled by this one mega corporation, it’s like, you know, a whole lot of resilience.
In a certain sense, we’re trying to take the Internet back. That’s the summary of why federation is important and stuff like that. Take the Internet back! That’s the best description I can give of that whole structural thing. Unless if I’m not addressing what you said.
Irina: So one of the other things I really wanted to ask is, it looks like you have a lot of contributors. Who are the people who work on MediaGoblin, and how can someone else get involved, and what’s your process and how do you manage that?
Christopher: So we have a lot of contributors to MediaGoblin. I was just updating the authors file in the project and it’s almost 75 people listed in that file at the moment. So, a lot of people.
Most of our collaboration happens on IRC, on a chatroom. We also have a mailing list and we have all the standard free and open-source software mailing lists, bug tracker, blah blah blah, but you know, not everyone who collaborates on the project is a developer, either. We do have developers and we also consider — and a lot of other contributors are also graphic designers, although we need a lot of more help with that right now. A lot of the help we have with people who were working on graphic design have been running out of time recently to be able to help.
We have a lot of help from people doing translations and so on. But if you want to get in, and you actually want to join us and actually start working on the project, just go to mediagoblin.org. There’s a ‘Join Us’ link on there. Just click that. It’s got information. And you can talk to us on our mailing list. But most especially a lot of the conversation happens in our chatroom. If you want to jump in there, I have a terrible username called ‘paroneayea’ on IRC, and I’m usually around and actually talking to people. And we just have a super friendly community of people who like to get people involved and working on stuff.
Irina: Great! That’s amazing that you’ve brought in all these contributors, and I’m actually kind of — this is probably a little off-topic — but I’m quite curious, like, how do you actually manage that in terms of the governance of code reviews, or who gets to do that, or what code gets in?
Christopher: Well, we have a lot of — so the general direction of the project is done in a couple of ways. I mean we do a lot of discussion on IRC and on mailing lists and bug trackers, and it’s just a lot of discussing and basically coming to agreement on what’s the best direction. In terms of governance, projects have that very authoritarian, problematic sounding, like, Benevolent Dictator for Life meme, right? And I guess I’m the Benevolent Dictator of the project, in terms of like, most of the decisions end up just happening within the community because people have discussed it and the community comes to a consensus, but I kind of set the general direction of the project and help guide the community in terms of where it’s going.
But I’m not the only person doing that. Like as the project’s gone on, oftentimes there are other people who just kind of have become the authority of it. We have a person who has kind of become the authority of the database level of things, somebody who’s kind of become the authority of the graphic design level of things. But I’m also the only person paid presently to work full-time on the project, so that means that I’m the official herder of cats. So, yep.
Irina: So it’s kind of your way, with these community discussions. So I think to wrap up, what is the future in the sense of what have you got planned for MediaGoblin going forward, and how can you make it easier for people who aren’t very techy or necessarily understand this very well to actually benefit from and use MediaGoblin?
Christopher: Right. So I’ll say that the next number of months ahead are going to be very interesting I think. This summer we had six students between Summer of Code and Outreach Program for Women who finished almost all of our goals for 1.0, but now we have this massive pile of code that we need to get cleanly integrated. And we’re very picky about getting clean code with MediaGoblin, so we’re going to have to work through all that and try to make sure that everything is actually merged in nicely.
But around that time, especially when we end up — do — get federation in, we’ll have a whole lot of new things landed. Search is actually coming soon, so you know, you can actually search across a site. Very shortly we’ll end up having an administration panel built in there. And that’s kind of boring stuff I guess, but the big feature of federation is the one big thing that when that lands we’ll finally hit 1.0.
Prior to 1.0 we focused on trying to make clean and beautiful software with the acknowledgement that it’s probably not going to — that you’re probably only free and open-source software nerds that are going to use it, but trying to design it in a way so that it’s not made for free and open-source software nerds, if that makes any sense whatsoever! But after 1.0 comes out, a big focus is actually going to be shifting from just building the features to make MediaGoblin fit that whole dream of what we laid out to actually getting people to be using MediaGoblin.
So, that shift will be happening soon. 1.0 should not be too far out. Of course it’s always hard to know. But I think we’ll be seeing some interesting things happening once that ends up happening. And hopefully we can start going on our massive crusade-y push to get everybody pushing over and you, your family, and friends all using MediaGoblin. It’ll be an interesting year, I think.
Irina: Ok, awesome! And I think on that note, we’ll wrap. Thank you very, very much, Chris! Really good to talk to you, and we’ll be in touch again soon. Bye!
Christopher: Yeah! Thanks for tolerating all my ramblings!