Mark Callaghan started a discussion on mysql replication lag today on the MySql@Facebook page. This happens to be one of my favorite topics – thanks to some related work i did at Netapp. I was happy to know that there’s already a bunch of stuff happening in this area for […]
I have been finding and reading some great references on flash memory lately and thought would collate up some of the better ones here (and leave some takeaways as well). For starters, ACM queue magazine had a great issue entitled Enterprise Flash Storage last year. Jim Gray’s and Goetz Graefe’s […]
One of the relatively late lessons I have received in operating a Hadoop cluster has been the (almost overwhelming) importance of compression in storage, computation and network transmission. One of the architectural questions is whether compression belongs to the file-system (and similarly the networking sub-system) or whether it is something […]
It’s incredible – but Rajeev Motwani is no more. It’s hard to imagine that i won’t run into him anymore walking University Avenue or that he’s no longer an email away to discuss (and tear down) the latest startup idea at some cafe nearby. My heart goes out to his […]
A formal recipe on running SQL queries using EC2 against S3 files is now posted at: http://wiki.apache.org/hadoop/Hive/HiveAws/HivingS3nRemotely But not before hitting a few more bugs ( HADOOP-5861 ). Running a TPCH query using Hive was a pretty high point. (I did have to omit the order by clauses though :-() […]
I have been enjoying my vacation time in India for the last few weeks and one of the fun projects i had taken up was getting a good story around running Hive on Amazon Infrastructure (AWS) . The use case i had in mind was something like this: A user […]
Curt Monash posted a blog post on our (myself and Ashish Thusoo’s) conversation with him regarding Hadoop and Hive and their deployment and usage at Facebook. It is heartening to see the mainstream database and analytics community starting to cover Hadoop and Hive. Even though these projects are rapidly becoming […]