Foller.me is Back Online, and now Filled with WordPress Goodness!

Twitter Analytics

Foller.me has got a new home, again! I’ve spent a few days working on a complete overhaul since Google App Engine’s pricing model has changed, which made the app consume over $5/week. It’s not big money but I thought I’d rather host the app next to my own blog for free ;)

The (yet another) new version of Foller.me is powered by WordPress, so if you’ve got friends who still think WordPress is for blogs only, share this with them. Instead of inventing a new user interface I went with Bootstrap and was able to prototype the whole thing really, really fast. I removed the followers geography part for now, I’ll need to find a better way to lay that out and make it actually function. Meanwhile you can enjoy two brand new sections.

New to Foller.me? Foller.me is a Twitter analytics application that gathers a bunch of interesting stuff from a public Twitter profile. Mike’s profile is a good place to start with ;)

Automated Serverside Tweeting Using OAuth

Great, our Foller.me Rundown account (@fmrd) now tweets via OAuth and has our application name and link attached to every tweet. It was indeed pretty simple. After reading the specs of the OAuth protocol I came up with a fairly simple solution. I’m using this open source Twitter OAuth php library by Abraham Williams which is quite good, and I do recommend you try some basic OAuth stuff (with sessions, based on Abraham’s example) before proceeding to automated tweeting.

Okay, let me first breifly explain how OAuth at Twitter is supposed to work (focusing on automated work). Step by step:

  1. You browse to some hidden area (which nobody but you has access to) and initiate the app registration process. At this point, your app should go ask Twitter for a request token and provide you with a link to Twitter authentication (which will contain the received token)
  2. Next, you click on that link which directs you to Twitter, login, click allow and you’ll be redirected back to your application page (not the hidden area!) with the request token attached to the URL.
  3. You copy that token, browse back to your hidden area and initiate the app validation process by providing the token in your request (GET)
  4. Your app will go talk to Twitter again asking them for an access token. It stores that token (in a very safe place) for later use.

That’s pretty much everything. Once you have your access token you can update your status via OAuth as much as you want. Also note that when I mention request token and access token, I mean request token secret and access token secret too. OAuth tokens come in pairs. Token + secret. Yes, it is that simple!

Now let’s get to some coding! Suppse your app is called MyApp and is located at myapp.com. Make sure your hidden area is actually hidden. Choose a nifty directory for your place and make sure you protected it with .htaccess (allow by IP or based on authentication, it’s up to you). Don’t worry, Twitter will not try to access that directory. Twitter (the OAuth Service Provider) doesn’t do anything but give responses to your requests, so block that as strong as possible.

Suppse your hidden server auth place is at myapp.com/hidden/ and there’s an index.php file, your requests would look like this:

myapp.com/hidden/?register

That would mean “initiate the OAuth registration process!”, which will give you the URL to Twitter. And:

myapp.com/hidden/?validate&oauth_token=whatever

Which will get your Twitter OAuth access token and store it somewhere safe. Remember that you’ll have to replace the word “whatever” with the request token provided by Twitter after you authorize.

Let’s look at the php code (omitting the includes and blah blah blah). Just read through the comments, you should be able to understand. Also make sure you got your $consumer_key and $consumer_secret setup in the Twitter OAuth applications settings.

if (isset($_GET["register"]))
{
    // If the "register" parameter is set we create a new TwitterOAuth object
    // and request a token
    $oauth = new TwitterOAuth($consumer_key, $consumer_secret);
    $request = $oauth->getRequestToken();

    $request_token = $request["oauth_token"];
    $request_token_secret = $request["oauth_token_secret"];

    // At this stage you should store the two request tokens somewhere.
    // Database or file, whatever. Just make sure it's safe and nobody can read it!
    // I'll dump mine into files using file_put_content:

    file_put_contents("request_token", $request_token);
    file_put_contents("request_token_secret", $request_token_secret);

    // Generate a request link and output it
    $request_link = $oauth->getAuthorizeURL($request);
    echo "Request here: <a href="" . $request_link . "">" . $request_link . "</a>";
    die();
}
elseif (isset($_GET["validate"]))
{
    // This is the validation part. At this point you should read the stored request
    // tokens. You'll need them to get your access tokens!
    // Mine are located in two files:

    $request_token = file_get_contents("request_token");
    $request_token_secret = file_get_contents("request_token_secret");

    // Initiate a new TwitterOAuth object. This time we provide them with more details:
    // The request token and the request token secret
    $oauth = new TwitterOAuth($consumer_key, $consumer_secret,
        $request_token, $request_token_secret);

    // Ask Twitter for an access token (and an access token secret)
    $request = $oauth->getAccessToken();

    // There we go
    $access_token = $request['oauth_token'];
    $access_token_secret = $request['oauth_token_secret'];

    // Now store the two tokens into another file (or database or whatever):
    file_put_contents("access_token", $access_token);
    file_put_contents("access_token_secret", $access_token_secret);

    // Great! Now we've got the access tokens stored.
    // Let's verify credentials and output the username.
    // Note that this time we're passing TwitterOAuth the access tokens.
    $oauth = new TwitterOAuth($consumer_key, $consumer_secret,
        $access_token, $access_token_secret);

    // Send an API request to verify credentials
    $credentials = $oauth->oAuthRequest(
        "https://twitter.com/account/verify_credentials.xml",
        array(), "GET"
    );

    // Parse the result (assuming you've got simplexml installed)
    $credentials = simplexml_load_string($credentials);

    // And finaly output some text
    echo "Access token saved! Authorized as @" . $credentials->screen_name;
    die();
}

That’s all. It wasn’t difficult, was it? Now that you’ve got your access tokens stored, you can call Twitter API using OAuth at any time! Here’s a brief example:

// Read the access tokens
$access_token = file_get_contents("path/to/access_token");
$access_token_secret = file_get_contents("path/to/access_token_secret");

// Initiate a TwitterOAuth using those access tokens
$oauth = new TwitterOAuth($consumer_key, $consumer_key_secret,
    $access_token, $access_token_secret);

// Post an update to Twitter via your application:
$oauth->OAuthRequest('https://twitter.com/statuses/update.xml',
    array('status' => "Hey! I'm posting via #OAuth!"), 'POST');

Then setup a cron job to access that page URL and you’ll be automatically tweeting! That’s about it.

Now, in conclusion, a few security suggestions. Never, NEVER place your tokens into a publicly visible folder. Deny all HTTP access to them via .htaccess (look at the Files directive) and yes, I’m going to let you finish, but please, PLEASE secure that hidden place we talked about earlier. Yes I’m repeating this and I’ll keep repeating it over and over. If hackers gain access to your hidden place, they’ll be able to swap your account with another one (or just break it). If they get to your access tokens, then they might spam through your account. That’s not very nice, is it? So please, IP based security, password protected, whatever. I close the whole directory down with a “deny from all” rule in .htaccess once I got my access tokens, so if for any reason I’d have to update or change them, I’d have to do more than just browse there.

That’s all. Have a good time with Twitter OAuth and I hope everything goes well. Feel free to post questions or any kind of feedback in the comments section.

Twitter API: Moving From Basic Auth to OAuth

As I mentioned earlier this week, with Foller.me beta 3, people now have the ability to follow tweeps directly from the website with a single click, without having to browse to their Twitter profile nor providing us with their Twitter credentials (thanks to OAuth. Read this post if you haven’t: The Importance of Using Twitter API via OAuth).

Now, why not take even more advantage of Twitter OAuth? As mentioned in the documentation and a few tweets by @netik (John Adams, Ops Engineer @ Twitter), due to the high growth of Twitter apps being developed every day, the source parameter in the statuses/update calls will no longer give you the desired result (attaching “via Your App Name” with a link to your website to the tweet). Calls with no source parameter come out as “via API”. Ones with unknown source parameters come out as “from web”. This doesn’t apply to already developped apps such as TweetDeck and Seesmic Desktop and they still use the source parameter via Basic Auth.

So how do I get my Twitter app name listed in the tweets?

And the answer is OAuth. Once you subscribe your app to the Applications Using Twitter page, those guys know about you. They know the name of your application and they know where to link if you post “via” your application. The key here is posting “via” your application. Well, the little “Tweet my profile” link at the bottom of a Foller.me profile (if you’re signed in) is fairly simple. We’ve got an authenticated (via OAuth) user and an OAuth request method:

$oauth->OAuthRequest('https://twitter.com/statuses/update.xml',
    array('status' => 'Test OAuth update. #testoauth'), 'POST');

That will post from the authenticated user via your application. Sweet isn’t it? But, you probably know our Foller.me Rundown feature, which tweets through the @fmrd account. It’s totally automated and uses Basic Auth to post. As I said above, Basic Auth will not give us the “from device” bit in your tweets, so we have to use OAuth. And this is actually what I am after.

There are a few request tokens and token secrets that travel between both servers (Twitter and the client) during OAuth authentication. In general OAuth usage, we store them into our user’s sessions on server. Now what if we store them into our database (or some other place) and when tweeting via @fmrd use THEM instead of starting a new Basic Auth session? This means that I somehow need to send myself (on a closed by .htaccess page or whatever) to the Twitter authentication page with a generated OAuth token, then, whenever Twitter redirects me back to my page, I need to copy the received “request token” and secret and write it down somewhere. I’ll have to dig deeper into OAuth for this, rather than just use a ready-to-go library that works with sessions. I’ll try this method out and write about it next week. It’d be cool if @fmrd could tweet “via Foller.me”.

Foller.me Got Coverage On Mashable!

The first thing I do when I wake up on an ordinary day is check my e-mail and Twitter, and today was an ordinary day until I met some strange messages regarding Foller.me in my inbox. Why is everybody suddenly so interested? Turns out that this day is not ordinary at all. Yes, Foller.me gets covered by Adam Ostrow at Mashable! The post became part of the Mashable’s Spark of Genius series via the Microsoft BizSpark programme. I came accross Microsoft BizSpark a few months ago but I weren’t too excited as all they had to offer was based on Microsoft technology while Foller.me is based on open source software.

But anyways, thank you Mashable and thank you Microsoft. We’re having a big party tonight!

Update: Due to the heavy traffic from Mashable and Twitter today, some profiles on Foller.me today turned out as “Not found”. I’m not quite sure about the technical reasons but I’m investigating certain profiles and prepairing to write a fix to make sure it doesn’t happen again. Sorry for all the inconvenience caused. After all this is my first ever Twitter app! It’ll get better, I promise!

Followup: Foller.me: The 404 Issue

The Importance of Using Twitter API via OAuth

I hope you noticed the latest changes at Foller.me. I’m talking about the new Followers rate section thanks to the TwitterCounter API and of course something I’ve been dreaming about since the launch of the project. You view a profile at Foller.me before making a decision about following that particular person or not, right? And yeah, we had a link at the bottom of the page that lead to their profile on Twitter, where you could click the follow button.

Now we’ve updated that section to a Twitter OAuth powered follow button. This means that once you authorize Foller.me to use your Twitter profile without having to even input your username or password, you can follow people directly from Foller.me, without having to do any extra clicks. Yeah, we’re ready to remove our beta label and as we promised we’re coming up with a few more features and optimizations.

Guess that’s enough for the news section. Now, back to the topic of this post. OAuth. Y’know at the very beginning I was thinking about giving people the chance to input their username and password on Foller.me, but hey, that’s dangerous, right? I still see tonnes of websites and Twitter services, which are super cool, and yes, they still use basic authentication instead of OAuth. Seriously, it took me less than two hours to incorporate OAuth into Foller.me and once somebody has authorized with you (on the server side) you’re able to do all the stuff with their account with no difference from baisc auth! No limitations at all! Please take a look at the Twitter OAuth Examples which include ready-to-use libraries (and classes) for the major programming languages including php, Python, Ruby, .NET and a bunch of others.

So, why bother switch to OAuth? Well, personally I hate websites and Twitter services that would ask me for my Twitter username and password, I start to think that they’re scam (don’t you?), even if they’re not. I repeat, I see tonnes of those, and I gave out my password only to a couple because I really, really wanted to see what’s inside. After that, I immediately picked a new password for my Twitter account. And yes, I really can’t wait till TweetDeck, Seesmic Desktop and the others implement OAuth into their apps. That would make them extra cool, seriously.

Here’s more! There’s also lots of discussion going on in the Twitter Development in Google Groups and I heard somebody mention that the source parameter for your apps will no longer be available sooner or later. Yep, they’re closing down the basic authentication method. I’m not sure when, and the Twitter API Wiki says that the date hasn’t been announced yet, but hey, you should do it now before it’s too late. OAuth applications won’t need any source parameter as Twitter already knows who they are after signing your app with them.

So dear friends, please switch your apps to OAuth, it’s very, VERY important.

Foller.me Featured on ProgrammableWeb.com

Wow this is so awesome. I received an e-mail this morning and here’s what it said:

Dear kovshenin,

Your mashup Foller.me has now been published on ProgrammableWeb.
You can see it here: <a href="http://www.programmableweb.com/mashup/foller.me">http://www.programmableweb.com/mashup/foller.me</a>

It is also our Mashup of the Day for 09/13/2009. Congratulations!
It will be on the front page of ProgrammableWeb today.

Thank you for submitting your application and sharing it with our community.

Best regards,
The ProgrammableWeb Team

Well I’m not sure if there’s anything else I should add. I’m so excited that we finally made it to ProgrammableWeb – the best resource around for APIs and Mashups! Hope to see our API there too in a couple of months, meanwhile feel free to rate us: Foller.me at ProgrammableWeb.com.

Update: Thank you, thank you, you’re far too kind! Heh, take a look at this tweet I got mentioned in a few hours ago:

@kovshenin, thanks for the note, we’ve just listed Foller.me API on ProgrammableWeb

That does it. Foller.me has now been listed on ProgrammableWeb’s API directory. Awesome! Check it out: Foller.me API at ProgrammableWeb.

Foller.me: Almost Ready to Remove the Beta Label

With a few core changes to Foller.me (hope you noticed the performance tuning) we’re almost ready to remove the beta label. I wrote on Twitter a few weeks ago that the beta label will be removed in September, so here we are. I’m not sure if it means anything to you, but it does mean a lot to me. You see, a product in beta is used with caution most of the time, that’s what I do. I like to play with betas but I don’t actually use them (except for Yahoo! Pipes and others of course). I think there’s something similar between the words “beta” and “startup”, so yes, I’d like to show the world that we’re stable. No glitches, no more server reboots and ready for massive hi-destructive traffic (thank you Amazon EC2!).

A few words about the core changes. We improved our caching mechanism – the relations are now being cached for no longer than one day. They’re used when generating the followers map, while the topics, mentions and hashtags clouds cache for only an hour. Same caching now applies to the Foller.me API, besides, client-side caching is always welcome. This has speeded up our overall performance and it seems to be running smooth for the past few weeks.

The Foller.me API though is still being improved. We’re coming up with a few more feature improvements to the REST API, most of them will involve styling the clouds and we’re also working on a few more examples to show you how to get the most out of the API goodies.

The beta label removal date hasn’t been announced yet, but we’re thinking September (you know, in Russia kids go to school 1st of September each year, so we’re hoping that their parents buy them brand new PCs and of course a broadband internet connection, therefore kids will discover Twitter and hopefully Foller.me ;).

Thank you for all your support and good luck!

Update: Foller.me is still beta ;) just not yet .. (12/22/09)

New WordPress Plugin: The Foller.me Widget

Here’s a way for you to show what you’ve been tweeting about lately. Right on your blog, in the sidebar. No Twitter passwords required, cause this thing works via the Foller.me API. This is the first ever app (widget, gadget, or whatever you wanna call that) based on our Foller.me API, so I hope we have a great start here, and you young developers out there, dig through this widget’s source code (which is GPL btw) and play around with the API. Perhaps your app’s the next, and please DO write to us if you do, cause you don’t wanna miss all the credit we offer ;)

The plugin’s located here: Foller.me Widget and hosted at the WordPress.org plugin directory right over here. Thank you for all your votes ;)

New WordPress Plugin: Twitter Followers Widgets

Hey, there’s a new widget that can display user pics of people you follow (and people who follow you) on Twitter. It updates whenever you gain a new follower.

That’s a new widget for WordPress I wrote a few days ago, and received aproval from the WordPress.org plugin directory today, so it’s the official launch. Go get yours right over here: Twitter Followers Widgets and don’t forget about the feedback! Yup, themes are cool, but widgets are way cooler! Also, as announced on the Foller.me blog today, we’ve finally released the public version of the Foller.me API so we’re gonna have more Twitter widgets developed next month, wohoo!

You may also dive into the development of your own stuff using the Foller.me API, it’s pretty simple, seriously, check out the API wiki and start making your own widgets! Because widgets are sexy ;)