Following people is a weak proxy
Posted by sam
I find myself these days hesitating to follow new people, no matter how interesting their writing is. Part of the problem is, for me, the internet is too strong. There is so much to read in my Google Reader subscriptions and Twitter stream that I have trouble not wandering off and failing to get work done.
But the bigger problem is noise in the stream. It's not a big deal if 40% of the posts are irrelevant, if you're only getting a few dozen a day. But if you're getting hundreds of updates a day delivered to you, and 40% of them are things you never want to read, or don't want to read in that context, then it starts to feel a lot like the bad old days of spam where you had to spend a substantial amount of time clearing out your inbox before you could even get to your mail.
The issue is that Twitter is confused about why we follow people (in fairness to Twitter, this has evolved over time). If you follow someone because you're friends with them, then you probably want to see all of their posts. But if you follow them because they sometimes post interesting links, then following is a weak proxy for what you really want. What you really want to follow is the interesting links part. Or maybe some subset of the interesting links, even. Ideally, I don't even want to have to click through to see if a link is relevant, I want the system to decide that for me, so the only things that get presented to me are things I'm likely to want to read.
I don't know if there are tools out there that do this - I doubt any do it well, if so. What I really want is something like Gmail implemented with priority inbox. But that took a pretty serious ML effort to get right (I know, I was involved with that team and hassled them to get it right - and it took a bunch of very smart people a while to really nail it). But I really want something like a Twitter stream with good markup on it (like auto-generated hashtags, and hashtag normalization, and ontology, so I can follow topics and get a good feed), and a feedback loop about what I'm likely to find interesting.
There's a limit to what people can read and digest every day - it can stretch but it's a fundamental limit at some point. So, over time, social networks have to optimize for relevance - I can only really read 50 or 100 articles a day, so they all need to be really, really high quality (this has interesting effects on writing over time, but that's a thought for another day).
Facebook Connect is a bad idea
Posted by sam
I don't like Facebook Connect.
As a developer, it's great - very, very easy to use, the Javascript library is great, all good. And as a user, I actually like it too - it's very fast and very consistent to log into new sites, which I do all the time. In fact, that's the problem. I think it's a phisher's paradise.
People have written about this before, of course. You could do "traditional" phishing by sending a "your friend wants you to see this" link that leads to a fake Facebook login form, and bam! you have someone's credentials. Nothing really new here, people have been doing that kind of phishing for a long time now.
There are two things that are different here, that I think make it potentially much, much worse. The first is the power that the Facebook credential is accumulating. This is really just starting, but every day I see more and more sites using it as a credential, and it's not hard to extrapolate to a world where most sites use it. Which means if you get phished, you are more vulnerable because so many other services will admit the phisher with your credentials. Getting phished on a single password right now just exposes you to whatever that service is. Getting phished on your Facebook password exposes you on services you've never even heard of, as long as they use Facebook Connect, plus any you've already signed up on. True, you can change the single Facebook password to be safe on all of them, but you're still handing a lot of power to the phisher, and you have to notice that you've been phished, which most people don't until it's too late.
The second thing that's different is that the Facebook APIs are designed to be run on the server as well as the client. And users are getting used to seeing sites (like this nice one I just saw today, likebutton.me), that ask for their Facebook credential so they can access a bit of your social data and do something interesting with it. If a site like that is malicious, it can easily serve you a fake Facebook login form, then do the real login on the server with the API (acting as a browser), and then, once they have the OAuth token, continue on functioning normally, showing you what you expect (as long as they don't try to do anything using the cookie that they never picked up). Also, if you do this server side, you can request all the power of the Facebook API for your app, including seeing all the user's friends data (because you can pretend to say "yes" when Facebook asks if it's ok for the app to have that power). This would look very suspicious if you actually asked the user for all this power legitimately.
The point of all this is that the Facebook Connect credentials are getting much more valuable all the time, the nature of Facebook Connect is such that there are many more opportunities for phishers to grab them, and Facebook Connect, by its very nature, is encouraging users to get very comfortable typing in their Facebook credentials on all kinds of sites, from high value brand names, to new things you've never heard of. If a big brand name uses it, it must be safe, right? So this little company I've never heard of, well, it must be OK to log in to them using it too.
I'm not saying that OAuth is itself unsecure or Facebook Connect's implementation of it is faulty. I can't imagine a new kind of attack, per se - you still have to get the user to type their email into a fake Facebook login form. But the more often you type in these credentials all over the web, the more likely you are to miss the one bad site, and the sites now have new tools to be less obviously bad. The only real defense is that Facebook (or some third party like Google) have to detect the sites that, every so often, show a bad login page to a user and grab their credentials. Sure, the site could be shut down by Facebook, but they have to notice it, and there will be a quick arms race to hide the fake auth page and use the real one whenever the bad site suspects that the request is coming from Facebook itself.
I really hope someone is watching. As Facebook Connect spreads, I think phishing is going to get much naster, spread much faster, and be much harder to spot as a user. That credential is just too tempting a target.
It’s too easy out there
Posted by sam
A bit of historical trivia about Google Docs...Before the Upstartle team started working on Writely (one of the first pieces of Google Docs), we worked on something else for about 8 months. What was this idea, forgotten to the mists of time? Drumroll...a distributed, peer-to-peer, bug database.
I know what you're thinking: "Brilliant idea! Why ever did you abandon something that only a few thousand geeks will even understand, much less use, in favor of building something that tens of millions of people use every day?". Actually, I'm sure it's more like "ech! What a stupid idea! Why am I still reading about this?"
The point is, it was a dumb idea, we wandered in the wilderness, and eventually (and pretty quickly) learned better and moved on. No one was very kind to the idea, and it died the death it deserved.
Now, imagine you're an angel investor in these frothy, competitive days. Or a VC, worried that the angels are going to put you out of a job, or whatever. You're talking to a team from YCombinator or Launchpad or one of the other incubators. They're young and fresh and excited about what they want to build, and even though it doesn't *quite* make sense to you, it seems to be making sense to 10 or 15 other guys who want in on the deal.
What do you tell them? "I don't get it (e.g. I'm old, don't take my money)", or "wow! you're the best thing ever!". YMMV, but I see a lot more of the latter going on these days.
Will it kill the little startups that might otherwise have found a good product and market? Maybe, but they're getting well funded, so it won't kill all of them, at least not right away. Is it the best, most honest, effective way to build products? Not really. It reminds me a lot of dot com days. Lots of people will waste time building small things that not enough people care about. They'll disappoint their investors and users as the products collapse. There will be tears when the investors start actually criticizing the business ideas. The brittle teams who talk about pivoting but don't have the stomach to kill a pet idea won't do as well as the teams who understand that success is always built on mistakes and failure. OK, that sounds preachy, but you get the idea.
The good news is, I'll let you know what the Restartle team (yep, the old Writely team) is working on soon, so you can tell me how dumb that one is. 
Twitter is the brain of silicon valley
Posted by sam
OK, weird thought time...
Remember all the theories about intelligence emerging from networks? For a while, people were looking everywhere for networks that had emergent or even "intelligent" behavior. There's a well formed theory of this, neural networking. A network is formally defined as a collection of nodes that take inputs, perform some weight computation on them, and produce an output.
If you hang out on Twitter at all, especially if you follow the "valley elite", you get a sense of this kind of network forming and being used. The nodes are the VCs, angels, entrepreneurs, followers and various other commenters. The connections between the nodes are the "following" relationships. Tweets are the neural signals between nodes along these connections. Each node follows a bunch of people/sources (these are the inbound dendrites), reads things over the day (computes internal state based on inbound signals), decides to write when inspired (reaches some internal threshold), and tweets some processed output (fires the axon). Obviously not all tweets are this thoughtful, but noise has a role in neural systems too.
We talk about groupthink in the industry but it's literally true in this sense. This network has some really interesting properties too - it's much more complex than a normal neural network, first because the nodes themselves are very complex neural nets, and second because the messages, even though short, are much richer than just a simple weight or yes/no signal. So this is kind of a meta-brain, a thinking network made up of thinking networks. The other interesting thing is that so many subnetworks are forming, analogous to the real human brain. Some are filled with noise most of the time ("I'm here!"), but even these are sometimes useful to the larger network ("OMG, someone just landed a plane in the Hudson").
What's particularly interesting about this to me is that I was just thinking a few days ago about how complex the industry is becoming. So much is happening in so many places, so many people are trying interesting experiments, that it's beginning to get hard to follow it all, much less process it thoughtfully. But the "brain of silicon valley" is adding real value here - if you position yourself correctly in the network of opinion and messages, you can benefit from nodes (people) upstream from you and follow a very complex set of topics reasonably well. It's not perfect, the compression is lossy even from the best commentators, so you have to be careful, but in a real sense, the brain of silicon valley is "thinking".
I wonder how this will evolve over time, as these networks get more developed, and people find ways to take advantage of the processing that's happening in them. What happens when something like Hunch gets it's teeth into these networks and is able to suggest really useful new connections between nodes, or to find signals that a specific node will find valuable? Some interesting companies out there, for sure...
People are better than the cloud
Posted by sam
5 years ago, when we were developing Writely, I heard something surprising. One big software company was very interested in us, specifically because "you can't pirate a web service". This was the concern of the time, that everyone was pirating all the desktop software. That turns out to be a nice feature of the cloud, but we've moved on since then, and that concern seems almost quaint. But I think there's another force coming that will be even more powerful, for anyone who makes software: people.
Allow me to explain. Today, I was re-installing iTunes 10, which has not been behaving well. I'm not much impressed with it these days, and I briefly debated going and looking for something else to play my music collection with. But I hesitated: my whole family uses iPods and iPads and iTunes...will I be condemned to a hell of incompatibility and pain if I do this? Better just suck it up and install iTunes.
I think we're going to see this more and more with socially enabled web services, and I think the world hasn't really found this model yet: services that incorporate your friends and family as *part of* the service. The holy grail of this approach is to make the switching costs of your product or application equivalent to the network-effect-induced switching costs of your social network. "I can't give that up because all my friends are using it" is a very, very powerful effect. As Facebook and Twitter and all the other networks get deeper and deeper into authentication and identity and get woven into all the web apps out there, I expect to see more of this approach. Zynga is flirting at the edges of it, but I don't think anyone has really gotten this right yet.
A face where there is no face
Posted by sam
Pareidolia is a well understood effect - humans have an amazing ability to see faces where there aren't any, hear voices where there aren't, etc. It makes sense - faces are a big, important part of our social world.
I've been wondering lately if there isn't a similar bug around social interactions and social authority...we're so tuned to understanding and seeing the social and political context we're in, that we can invent social relationships where there aren't any, based on ambiguous signals (like chance occurrences or coincidence).
This makes me wonder if this is why we have religion and everyone imagines their social relationship with some kind of god, or nature spirit or whatever. We're seeing a social relationship in a noisy signal, where there is none.
how to fail in large groups
Posted by sam
All good ideas look dumb at first. Unfortunately, most of the dumb ideas also look dumb at first ( the worst case is the dumb ideas that don't look dumb, but that's another blog post).
OK, wisdom, but it's useless, right? How do you separate the really dumb from the good-but-seems dumb? Everyone (well, everyone in Silicon Valley) knows the answer: build something quickly to test your idea, and then iterate it in front of your customers. It's gospel, you can read this from any VC, Angel or Founder out there.
So, why don't big companies like Google manage to innovate as well as smaller ones? They have resources, and it seems like a simple recipe. But it's very, very rare for a large organization to be as innovative, consistently, as the little startups around them are. Is it just numbers? I don't think so.
The problem is that once you get a big enough group of people working together (any people, not just corporate types), they self re-inforce whatever idea they are working on. "Hey, 50 us are working on this, it must be a good idea, right?". This dilemma can't be overcome - anything that big needs coherence to succeed, but that coherence precludes effectively questioning the status quo. Cognitive dissonance rules.
It's very hard in that context to express doubt. As as startup founder, I can decide that what we're doing doesn't make sense and debate it with my entire "org" (e.g. the other two people I work with) on a daily basis, and it's fine - we actually make progress and move forward, and it doesn't drive anyone nuts - this is what we're in it for. But if you try that with 50 people, the communication overhead and inefficiencies will turn it into mush - everyone will either ignore the debate and do what they were doing (which is probably not the great idea you want, even if it's a good idea), or they'll be confused, decide the direction or leadership doesn't make sense, and give up, wander off into some other part of the company, or leave. There's literally no way to do product/market fit iteration in a group this size - it must be a smaller group that finds the vector. Once found, the larger group can't be beat for scaling the approach.
The best big companies try to keep this culture of iteration alive, but ultimately, it's impossible. Eventually the market circumstances will change sufficiently that the broad approach you are taking is no longer effective. That next "dumb idea" that eats your business or changes your direction won't make sense to the larger organization, and some pesky small thing will come along to do it instead. The only answer I see is to try to be as innovative in your culture as you can, and try to buy promising new companies at that magical stage where they've seemingly proven their dumb idea to be not so dumb, but haven't yet really scaled it up (you can buy it after it scales, but that's more expensive).