Software Rendering, Filesystems

Core i7 beats Intel IGP in DirectX 10 software rasterizer : I am very excited about the upcoming Larrabee GPU (or rather, hybrid-GPU) Intel is working on. I never found it particularly interesting to be restricted to a set of APIs for for defining and drawing scenes, as opposed to the good old days where it was all about relying on optimization techniques and clever programming to get get the most out of a pure software based rasterizer. Nowadays, at least on PCs and most game consoles, you must use either DirectX or OpenGL which provide a set of benefits (everything is taken care for you by the the GPU, the driver and the API implementation layers, etc) but also take away the fun. There is a multitude of reasons why the existing model works, but one could argue that innovation and advancement of the technology is hindered by being bound to a constrained environment and set of interfaces. I can't wait to see what near-future Carmacks, Sweeneys and Abrashes will do with the return to software rendering made possible by Larrabee and new, similar products by Nvidia and AMD.
Related references: Twilight of the GPU: an epic interview with Tim Sweeney, RAD Game Tools's advanced software rasterizer for x86, Michael Abrash, legendary x86 assembly and code optimization programmer, Software Rendering on Wikipedia, Nvidia's David Kirk on CUDA, CPUs and GPUs

Migrating to ext4 : We are looking into switching to ext4 filesystem for a few nodes on our 'testbed' environment now that ext4dev is considered stable enough to be renamed to ext4. ext3 has been sufficiently stable and performs well for our data set. Hopefully ext4 will be better in both aspects. We put XFS and ReiserFS to the test a few years ago and that didn't work out very well, though XFS, at least in paper, is impressive. Sooner or later we will need to work on our own file system, which would introduce a great number of benefits to our environment and would be fun to build.
Related references: Google File System, Lustre Filesystem

Saturday, 29 November 2008 2:03 pm

Enterprise level mashups

We are seriously considering offloading some of our data storage and hosting needs to Amazon Web Services, and if that works out - it most certainly will - we 'll offload some more and take it from there.

Amazon has quietly been building a google-grade, highly scalable, reliable and distributed in nature infrastructure, while most folks would disregarding it as an e-tailer. The largest e-tailer, but 'just' an e-tailer nevertheless. They were wrong. Bezos is not to be taken lightly.

Secure Storage Service(S3) is the AWS's flagship service. In short, it allows for 'unlimited' data storage and access of arbitrary data objects ( AKA files ) consolidated into buckets ( psudo-directories ). Its trivial to access the service via its XML (REST and SOAP ) based API. Its super-cheap ( especially if you happen to be dealing with the kind of price tags ISPs in Greece attach to their related offerings ) and, how about that, it works.

It's been suggested before and it gets more obvious by the day. With a very little sum of money, lots of will and dedication and, preferably, with a good idea to ago along, a bunch of guys can build the next Youtube. Delegate the storage and hosting to AWS, setup a few servers ( you can 'hire' virtual servers on AWS's Elastic Compute Cloud for 10 cents / hour ) to do your bidding, use blogs for free ( and highly effective ) PR, and hey-presto, money and fame are pouring in.

Google, Yahoo, Microsoft and an ever increasing companies are offering similar services ( software as a service, where did I here that before ) which you can bring in together, mash-them-up and you are set. Scalability, distribution and reliability is implicitly guaranteed by the service providers. We are entering this business sooner or later. All our services ( we love modules and components here ) have been designed to be accessible via XML (XML-RPC, REST, SOAP, JSON etc) . Also, we are big on clusters and fault tolerant systems so just about everything is already in place. In fact, everything has been in place for over 5 or so years, since we switched to the new architecture model.

We live in interesting times. Interesting things happen all the time. Go ahead, build the next big thing and contribute to that interest factor. Trust me, you are going to have a whole lot of fun doing it.

Saturday, 16 June 2007 6:52 pm

Linux Kernel Development : Book : mini review

Thanks to the wonderful benefits of weekends ( way more free time than than monday-friday avails us ) available to most of us, I mastered enough time to complete reading the book I mentioned in a previous post. All in all, its a great book. It provides just about everything one needs to start hacking the linux kernel, some unrelated by always useful and/or funny remarks by the author, and, above all, provides you with plenty of ideas you can use in your user-space applications and for motivation to refactor your code and algorithms to achieve the kind of simplicity, beauty and elegance the linux kernel 'gods' developers have achieved. Highly recommended.

I have three more books related to the subject which seem like perfect candidates for investing the remaining time ( until I will have to succumb to sleep, that is ) of the day. As always, do yourself a favor. Turn off the TV, go read a book. No matter what kind of book it is, no matter what kind of show you are watching a TV, it is always worth it.

Sunday, 15 October 2006 9:44 pm