Tweet! Tweet! Need to Integrate Twitter feeds into Liferay?

Twitter is now ubiquitous in the social landscape. More and more organizations are showing twitter feeds in their external facing sites. In a recent project, I was tasked with the integration of Twitter tweets into an Enterprise Portal. In this blog post, I am going to walk through the steps for installing and setting up the twitter portlet from the Liferay Marketplace. As you explore the twitter portlet, you will probably face the same limitations that I did during this project. Fear not, as I will also provide my solution to the limitations that I faced.

1. Copy the lkg (plug-in) file into your deploy folder of the Liferay bundle.
2. Start your Tomcat server and login in to your Liferay instance.
3. Select Add->Social->Twitter, add it to your public or private page.
4. Twitter Portlet will let you set up a Twitter account, this will go to Liferay component which is under Control Panel->My Account->Social Network. Here you can configure Twitter Account (Your Twitter Screen Name). For example, my Twitter screen name is RujuanXing.
5. Go back to the page you’ve added Twitter portlet. It’ll say “Your activities will be shown in Activities Portlet”.
6. Now you need to add Activities Portlet which is under Social category in your portal. This is where you will see the tweets of the user that you configured.

1. When you deploy twitter portlet plug-in, a table called twitter_feed is created. The table is used to store twitter screen names that you configure in Control Panel.

2. Twitter portlet schedules a job to fetch tweets from Twitter REST API. The job is configured in liferay-portlet.xml

3. When the user that you configured in the Control Panel tweets in Twitter, the Twitter Portlet will get the data and store the tweet in socialactivity table which is a default table in Liferay.

4. The Activities portlet grabs the data from the socialactivity table and displays it.

I will let you absorb the above information, and think about the constraints that you are facing with this particular offering of the Liferay Marketplace.

Some of these are obvious, some are not. Here are the items that I ran into:

1. The Twitter portlet available on Liferay Marketplace is using the Twitter REST API. Twitter has limited the REST API on the number of calls allowed. Unauthenticated user calls are permitted 150 requests per hour. Authenticated calls are permitted 350 requests per hour. For more details, please refer to https://dev.twitter.com/docs/rate-limiting/1.1. This becomes a key issue on a public facing site which could possibly have 1000s of users.
2. The REST API call is hard coded in the class. Whenever Twitter changes its API (which happens more often than you can imagine), you will need to change the code and compile and deploy. I faced this issue right at the outset. The twitter portlet that I was using from Marketplace, it uses legacy Twitter REST API to fetch tweets from Twitter.com. In March of 2012, Twitter announced their new API and in early October 2012, they turned off old endpoints. Since our project uses Liferay EE 6.1.10, I could not use the latest version of Twitter portlet which was targeted for Liferay EE 6.1.20. I did change the code and compile and redeploy. If only we could put the API in properties file, we will be spared the trouble of changing code, compile and deploy.
3. One more limitation that I observed, the marketplace twitter plug-in does not fetch the tweets when a user retweets other user’s tweets.
4. If you want to configure two or more twitter accounts in the twitter portlet, you are out of luck. I found this to be a serious limitation for my requirement.
5. All the tweets are shown in Activities portlet. Any social activity such as blogging will also show up on the Activities portlet effectively crowding out the tweets. There was no means of configuring a filter for just the tweets.

Because of these shortcomings, I decided to develop our own twitter-portlet for this particular project.

Below you will see the approach I took to solve the shortcomings I faced.

Twitter provides another API: Streaming API which is long live connection to Twitter. Whenever our subscribed twitter account tweets, Twitter will push the tweets to us. For more info, refer to https://dev.twitter.com/docs/streaming-apis. When you use REST API, tweets are stored at Twitter. With Streaming API, you need a place to store those tweets. In short, we need to design some tables in the database to store those tweets.

For this project, I designed three tables: organizationstatus, tweet and feed.

organizationstatus table: This table was used to store twitter screen name and organization id which is configured in Liferay.

feed table: This table was used to store twitter user id and organization id.

tweet table: This table was used to store tweets from the subscribed users on Twitter.

There are three specific applications: twitter-persistence, twitter standalone app and twitter portlet.

Twitter Persistence: This app is used to perform all database operations.

Twitter Standalone: This app is used to set up a long live connection to Twitter.com. It listens to a list of twitter accounts. When those accounts tweet on Twitter, those tweets will be pushed by Twitter.com to our connection. Our standalone app will store those tweets in tweet table. Because our listening list is dynamic, when user adds/updates/deletes organization’s twitter account in Liferay, we need to change our listening list. In order to do that, we have a job to check organizationstatus table every 30 minutes (you can specify the time you want). When the job finds that the table is changed, it’ll make the corresponding change into feed table including get the new twitter user id and then change the listening list.

Twitter Portlet: It has two functions: Primary function is to display tweets and the other is actually a hook which stores organization’s twitter screen into organizationstatus table. Whenever a user logs into Liferay Portal, the portlet will get the User’s organization, and display the tweets. If there are no tweets with current organization, the portlet will display the organization’s parent organization’s tweets. When admin adds/updates/deletes organization’s twitter account, we make a corresponding update to the organizationstatus table.

Second, how can we set up a long live connection to Twitter.com? Previously, I was thinking I need to start coding from scratch. But then I found twitter4j which is an unofficial Java Library for Twitter API. You can download jar files and demo here.

I was able to overcome the limitations that I faced while using the Liferay’s twitter portlet by essentially writing my own twitter portlet. With a clean design approach, and efficient coding standards, I was able to make the portlet do what I specifically started out to do.

Let me share a small demo with you. This demo is used to get the tweets which we specify in the filter. In this case, I am only listening to the tweets of RujuanXing and CNN.

Another question is how to get twitter userId based on twitter screen name. It’s really easy to do that when we use twitter4j.

Here is a screenshot of the twitter portlet that I developed –

Hopefully this helps in getting you started in writing your own twitter portlet! If you have any question, need help or want to share your perspective, just comment at the bottom of this post or contact XTIVIA, my employer!