About / Open Source

Twitter loves open source.

Twitter is built on open-source software—here are the projects we have released or contribute to. Also see our engineering blog for more details.

Want to work on stuff like this? Check out our jobs.

Ruby

cassandra.gem »

A client for the Cassandra distributed database.

zookeeper.gem »

An interface to the Zookeeper distributed coordination server.

contributions to memcached.gem »

An interface to the libmemcached C client.

twitter-text.gem »

Text processing routines for Tweets.

thrift_client.gem »

A Thrift client wrapper that encapsulates some common failover behavior.

cache_money »

A write-through and read-through caching library for ActiveRecord.

kestrel-client.gem »

A client for the Kestrel message queue.

flockdb-client.gem »

A Ruby Client for FlockDB.

simple_uuid.gem »

Simple, performant UUID generation.

scribe.gem »

A client for the Scribe distributed log server.

contributions to cassandra_object »

An object mapper for Cassandra.

contributions to Avro »

A serialization system.

contributions to piglet »

A DSL for writing Pig scripts in Ruby

Scala

kestrel »

A reliable, uncoordinated, loosely ordered message queue.

scala_json »

JSON toolkit based on Martin Odersky's Programming Scala book.

querulous »

An agreeable way to talk to a JDBC database.

flockdb »

A distributed, fault-tolerant graph database.

gizzard »

A framework for creating distributed datastores.

ostrich »

A library for collecting and reporting application statistics from Scala.

scala_build »

Ant and Ivy scripts to automate building Scala projects.

trie »

A trie data structure implementation.

snowflake »

A network service for high scale unique ID generation.

scala-zookeeper-client »

A ZooKeeper client library in Scala.

rpc-client »

A scala library that encapsulates RPC communications.

standard-project »

A more useful sbt project plugin.

killdeer »

A server for replaying responses to recreate production response characteristics.

util »

Miscellaneous Idiomatic Scala Utilities & Wrappers

contributions to haplocheirus »

Redis-backed timeline storage

Java

contributions to cassandra »

A highly scalable second-generation distributed database.

integer_list_input_format »

Codazzo's MultiRowInputFormat, adapted for integer lists.

twitter_text_java »

Text processing routines for Tweets.

contributions to hadoop »

A platform for analyzing large data sets.

hadoop_lzo »

Splittable LZO compression for Hadoop.

elephant-bird »

Our collection of LZO and Protocol Buffer-related Hadoop, Pig, and HBase code.

contributions to pig »

A platform for analyzing large data sets.

contributions to lucene »

A high-performance, full-featured text search engine library.

C/C++

contributions to scribe »

A scalable log aggregation server.

contributions to ree »

A server-oriented branch of Ruby.

contributions to mod_memcache_block »

An Apache module for applying IP blocklists stored in memcached.

contributions to thrift »

A framework for cross-language services.

JavaScript

twitter-text-js »

Text processing routines for Tweets.

Tools

murder »

Scalable code deployment with Bittorrent.

pig.tmbundle »

Textmate syntax highlighting for Pig.

jvm_gc_stats »

A script to parse and graph the JVM's GC trace output.

twurl »

OAuth-enabled curl for the Twitter API.

twitcher »

A tool for executing scripts when ZooKeeper nodes change.

gizzmo »

Gizzard command line client.