When Worlds Collide

Oct 13, 2009 in Business, Computing

0

Rich DeMillo is blogging.

He is writing about the interesting things that happen when technology innovation and business execution impact each other.  How technologists can get things done in a technology company and what business managers need to know to be successful.  When Worlds Collide is about how organizations can succeed when business execution and technology innovation seem to be on a collision course.  The "hey, you got your peanut butter on my chocolate" result.

Rich has an impressive background.  He was in charge of the rebirth of undergraduate education in computer science at Georgia Tech, Chief Technology Officer at Hewlett-Packard, VP of Computer Science Research at Bellcore, and Director of Computing Research at the National Science Foundation.  I had the chance to meet him at a football game a few years back.  He is nice guy.  And much easier to talk to than his background might suggest.

I love the title, When Worlds Collide.  The articles are thoughtful.  And I expect them to be as interesting and thought provoking as Rich's “Murder, Starvation, & Catastrophe – What Eric The Red Can Teach Us About 21st Century Innovation” that he delivered a few years back.  Rich was great in person.  I am looking forward to what he has to say online.

Government’s Gone Cloud

Aug 12, 2009 in Computing, Open Source, Presentations

1

This is a guest post by John Willis and Curtis Hill of Zabovo  John runs the AWSome Cloud Computing meetup group which the ATDC is proud to host.  Today John is speaking at the Military Open Source Software Conference at the Georgia Tech Research Institute. This article is based on his presentation. 

Last week the first sentence of an article in the InformationWeek periodical specifically targeted at IT employees of the U.S. Government read as follows:

This dry and seemingly innocent statement is in reality a blockbuster, a headliner worthy of amazement possibly and further investigation surely.  Any computer industry veteran with federal government dealings will tell you the phrase ‘U.S. government technology innovation’ is an oxymoron (with the notable exceptions of the DOD and NASA).  And now - low and behold! – the stodgiest of the stodgy is rapidly moving (that’s correct – rapidly) past all but the most innovative organizations in the world into the era of cloud computing!  I’ll throw in a few cloud basics in a minute but just for those who aren’t ‘in on’ the cloud computing debate many still question whether cloud computing is a transformational advance in computing as many others claim or just a horribly overblown catch phrase meaning little.   The InformationWeek statement is a really big nail in the coffin of the catch phrase camp.   It’s not the first and it won’t be the last.

Why the sudden change after years of being exactly the slow moving inefficient bureaucracy all small government proponents complain about.  Surely even the victory of a candidate who campaigned on upgrading the nation’s technology infrastructure could not have caused such a swift change in culture in 7 months.  Maybe this technology innovation is too compelling to ignore even for this historically laggard organization.  Maybe both.  Neither seems likely but here we have it.

Let’s look at a few basic definitions to make sure everybody is on the same page.  To follow the theme I will take this information from the National Institute of Standards and Technology (NIST):

Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. This cloud model promotes availability and is composed of five essential characteristics, three delivery models, and four deployment models.

Private enterprise is still trying to achieve a collective understanding of cloud computing and its value if any.  Many continue to argue over the viability of clouds for any but a few unique applications (massive data, map/reduce, etc.), claiming obstacles ranging from security to audit ability to performance.   After investigating these issues for many months, NIST has moved past these arguments and published a very mature definition of cloud computing, and an even more mature document on how to efficiently and securely implement a cloud computing environment.   The document examines each advantage and each challenge presented by cloud computing.  NIST sees the huge advantages many others see but they have also taken a hard look at the challenges and they view them as difficult but solvable.  NIST is even helping drive some of the standards in area such as security.

In another fascinating development a fellow by the name of Vivek Kundra was tapped by Obama as the Nation’s first Chief Information Officer.  Before taking the federal CIO role Kundra was the CIO for the city of Washington, D. C.   The punch line?  In less than two years Kundra moved the city to Google Apps replacing Microsoft Office software with Google’s SaaS offering.  The city now posts their procurement process on You Tube.  Kundra launched the ‘Apps For Democracy’ WEB 2.0 style collaboration contest in hopes of giving citizens a portal into such government information as crime reports and pothole repair schedules .   Kundra expected maybe 10 apps and got 47 in 30 days.  This endeavor saved the city ~$2.6M even after the $50K prize money was paid.  Kundra works by the mantra that citizens are "co-creators rather than subjects."   This 34 year old Gen-Xer is now in charge of information technology for the whole government!  Looks like this may no longer be your grandfather’s federal government – at least in IT.

All of the primary cloud vendors have stepped up their efforts to sell this latest technology to the government since they too see the sleeping giant awakening.   Companies like Amazon and Google who had done little business with the government in the past are now pushing the adoption of their cloud based technologies in all parts of the government.  SalesForce.com, possibly the most successful SaaS, has also sold its software to several branches of the government.   The DOD and NASA have both adopted elements of cloud computing.  NASA uses SalesForce.com and the open source cloud software Eucalyptus.  DOD is very actively promoting cloud computing internally.

The most innovative companies in the world today have provided the missing pieces needed to launch cloud computing as the NEXT BIG THING.  Even the traditionally sloth like U. S. government is on board.  To be sure, many of the truly transformational changes cloud computing will drive won’t be possible until the various technologies collectively dubbed ‘cloud computing’ mature even further.  Additional innovations (some already envisioned, some not) such as better security, improved virtualization management and real cloud interoperability are needed to speed cloud adoption rates.  Nonetheless, cloud computing is a transformational advance in technology on the level of the worldwide web or PC’s – or even bigger.  If you don’t agree and you are a CIO you should look for a career ‘Plan B’.      

Map Reduce for the People

Apr 09, 2009 in Computing, Internet

12

This is a guest post by Russell Jurney, a technologist and serial entrepreneur.  His new startup, Cloud Stenography, will launch later this year.  The article is an extension of a simple question on Twitter asking the importance of Map Reduce.  Some subjects take much more than 140 characters.

The Technical Situation in Brief

The advent of the personal computer and the Visicalc spreadsheet were the foundation for a revolution in computing, business and life whereby normal people could carry out sophisticated accounting, analysis and forecasting to inform their decisions to arrive at more positive outcomes. As Moore’s law has progressed and processors have become faster, and computers inter-networked, large volumes of highly granular data have been collected. Analysis of terabyte datasets on the same level as a spreadsheet has been limited by the disparity of acceleration between processor speed and computer I/O (input/output) operations. Intel has produced ever faster processor clock speeds without accompanying disk, RAM or bus speeds. Put simply: We have cheap and numerous computing resources and abundant data, but bringing those resources to bear on that data to generate real value from it has proven exceedingly difficult.

The widespread use of relational databases to access data in pre-defined static relationships has also limited our ability to discover and infer new and unique relationships among data. Dynamic analysis of large volumes of data in relational databases requires exhaustive pre-calculation of indexes and summaries of data for each relationship, and scaling relational databases to handle large datasets is a complex, painful and expensive process. As a result business intelligence systems relying on relational databases are prohibitively complex and expensive. Other methods of raw parallel computation, such as MPI, were exceedingly difficult. Such ‘smart kid only’ technologies have significant barriers of entry for mere mortals. In fact, multi-threaded, shared-memory computation in languages like C++ are considered some of the most difficult, arcane areas of computer science, leading to entire languages aimed at making concurrency easier.

MapReduce As the Way Forward

In order to extract value from large piles of data, we must escape the bounds of IO by going parallel and having many processors work on the data at once, without grinding our development to a halt dealing with complex algorithms and frameworks. MapReduce and platforms that implement it satisfy this requirement for a surprisingly broad set of problems. MapReduce is a simple way to process data in parallel among many commodity machines. You are already familiar with the power of MapReduce in your daily use of it - it is the pattern pioneered by Google to bring you the effective search on which we now all depend.

MapReduce is the design pattern that in combination with recent developments in cloud computing and cheap, plentiful broadband will bring us spreadsheet-style analysis of vast amounts of data ill suited to traditional database management systems in both scale and structure. MapReduce offers a cost-effective way for any business to harness massive amounts of computational power in the cloud for short periods of time to perform complex computations on large volumes of data that would be prohibitively expensive and time consuming on an individual machine, or that would require the construction of a data center to handle.

The Business Impact

What does this mean for your business? Knowledge of MapReduce has spread beyond Google, and it is now used by an increasing number of companies to extract value from web-scale data. Facebook, Yahoo, Cloudera and many others have embraced MapReduce in the form of Apache Hadoop, the platform around which most open discussion of MapReduce has occurred. As a result, a new generation of startups is rising that will take advantage of MapReduce to bring the same power that google pioneered on search to bear on a variety of datasets. New opportunities exist by ‘thinking big’ and extracting value from ever-increasing streams and volumes of data.

Example 1: Proving Global Warming

What does this really mean? It means that developers will have a clear way to reduce vast datasets to scales they can work with to extract information to inform your decisions. In this example from Cloudera, Hadoop and Pig are used to query a 138GB log of weather history for the last 100 years from the National Climatic Data Center to reduce that vast data to a scale the developer is comfortable working with. The result is this chart:

As a pile of data, the NCDC log informs nothing. When queried via map/reduce using Hadoop and Pig, we arrive at an informative chart that shows us an important trend. Would that chart inform a discussion about global warming? If you could get such clear visualizations about every minutiae of your business critical to your success, would it inform your decisions? Can you log and mine more data to streamline your operations?

Example 2: A Supercomputer for Every Biologist

When Amazon S3, EC2 and MapReduce via Hadoop are applied to the RMAP algorithm of genetic analysis, thanks to the work of one grad student, the result is a point-click supercomputer for every biologist that wants one in the form of Cloudburst for Amazon Elastic Map Reduce. Now any biologist that wants a supercomputer for this kind of genetic analysis can have one by the hour, and its as easy as point-click. More map/reduce genetic analysis algorithms are sure to follow. That's revolutionary.

Conclusion

We are constrained in our strategies by what we imagine possible. MapReduce and cloud computing open broad possibilities and business opportunities by placing a usable supercomputer by the hour in the hands of every startup that wants one. There is no problem which you lack the processing power to solve, its just a question of whether the hourly cost is profitable. That's a profound change from being bound to one machine. As a result of this shift, smaller companies can attack 'bigger' problems without a large up-front investment in hardware or software infrastructure.

A new renaissance in computing is coming that will be comparable to the business adoption of the personal computer and VisiCalc, and MapReduce will drive it.

Using LMRC To Pick NCAA Basketball Tournament Bracket

Mar 16, 2009 in Computing, Fun, Sports

3

Last year I wrote about Logistic Regression Markov Chain (LMRC).  LMRC is a Markov Chain that is purpose built as a college basketball ranking system based on basic scoreboard data.  LMRC is a tool to that can be used to help you fill out your NCAA basketball tournament bracket.  LRMC was created by Dr. Joel Sokol and Dr. Paul Kvam, professors at Georgia Tech's H. Milton Stewart School of Industrial and Systems Engineering.  It is now maintained, updated, and improved by Dr. Sokol and Dr. George Nemhauser. 

LMRC is right more often then other ranking methods and effective at sorting out the top teams in the later rounds.  Here is Sokol's and Nemhauser's presentation that highlights the power of the methodology. 

I used LMRC to pick my Final Four this year. Louisville, Memphis, Pittsburgh, and North Carolina were the result. Below is a pure play LMRC bracket up to the Final Four.  LMRC puts Memphis and North Carolina in the final with the Tar Heels prevailing... shudder the thought.  So I went had to go with my heart and my hope once LMRC delivered the Final Four.

Having been weaned on Louisville basketball at Freedom Hall, I took the Cards over Memphis in the semis and again over North Carolina in the final. 76 - 68. Terrance Williams gets the most outstanding player.

Go Cards!

Cool Air

Jul 31, 2008 in Computing

3

My MacBook Air was experiencing a core shutdown.  This was my fix.

Logistic Regression Markov Chain

Apr 10, 2008 in Computing, Sports

1

What is Logistic Regression Markov Chain (LMRC) you may ask?  It is a tool that can be used to help with selecting and seeding the NCAA Tournament field.  Or if the NCAA does not want to use it for that perhaps you can use it to win your office pool.

According to this article LRMC has predicted 30 of the past 36 Final Four teams correctly.  That's pretty impressive.  This year it predicted both the Final Four and Kansas as champion of the tournament. 

Something to keep in mind when putting together your brackets next March.  And I am looking forward to next year.  On a Sportscenter broadcast Dicky V had the Louisville Cardinals coming out of the gate as number three.

Seduced

Mar 24, 2008 in Computing

5

I have been.  And I gave in.

I bought a MacBook Air.  Perhaps not the most rational decision in the world.  But Paul Stamatiou (his review) walking into a meeting with one pushed me over the edge.  I purchased a gently used 1.6 GHz model on eBay for less than $1,500.

I asked my readers if I should get a MacBook Pro or the Air when it was launched.  Surprisingly, most of the hackers who responded suggested getting the Macbook.  The plain MacBook sans the Pro.  And it is indeed has the best performance/price ratio.  That would be the most rational choice for many.

But an equal number of people suggested I go with the Air.  My friend Joe Reger opined; "I've seen you out and about with a smaller laptop and have to believe that the portability and form factor is important to you."  Joe knows me well.  What it came down to is I did not want to increase the weight and footprint of what I carry around all the time.  Given that I have a primary machine that I use for the heavy lifting of photo editing and mostly use a laptop for Internet communications, web apps, and Office, the Air has plenty of juice for me.

And on top of that my kids think it's cool.

Macworld Keynote in 60 Seconds

Jan 16, 2008 in Computing

0

While I was staying up late last night contemplating if the MacBook Air was worthy, I was also watching/listening the Steve Job's keynote address.  If you don't have the 90 minutes to spare this clip by Veronica Belmont hits all the highlights in about a minute.

To Air is Apple

Jan 16, 2008 in Computing

11

I purchased my PowerBook G4 1.0 12" back in 2004 after it was discontinued in April of that year.  It's about the size of an 8.5 by 11 sheet of paper and easily weighs less then five pounds.  It was packed with everything I wanted in a laptop.  Two USB ports, a FireWire port, a modem port, Ethernet port, WiFI, and Bluetooth.  At the time the 40GB hard drive seemed huge for an ultra-portable.  It was a deal at about $1,200.  Somewhere along the line I maxed out the DRAM at 1.25GB.  Great machine.  It is my seventh laptop and the best one I have ever owned.  It literally is my notebook.  I carry it almost everyhere.

But that 40GB drive and 1.25 RAM are starting to get a little long in the tooth.  So I have been waiting.  And waiting.  Waiting until today, when Apple would introduce its new product lineup.  Let me tell you, I have been jonesing for a new laptop since last summer.  Have the money set aside to buy it.  I was hoping, really really hoping that Steveo would introduce the worthy replacement to my little friendly G4.

And I when I first laid eyes on the Air I thought that he did.  It is a stunning, stunning piece of technology.  Deserves a spot in MOMA.  It is beautiful.  Fake Steve would be proud.

But is it functional?

Some folks have claimed that it's a little expensive, but the price point does not bother me. 

Some folks have claimed that it is underpowered, but the specs do not bother me (I have a 2.4GHz 4GB memory iMac for video/photo editing and game on a console). 

What bothers me is that to make the Air beautiful they removed all the holes and seams.  It needs holes and seams.  No Ethernet port?  One USB port(which is claimed by the Ethernet adopter they sell for $29)(and a big BTW, I have never understood the whole Apple dongle thing, Steve must only like certain types of holes)? And battery that cannot be replaced by the user.

Yes, the Air comes with a battery that cannot be replaced by the user.  Perhaps another Apple first.  The cost of the battery is not that important.  I am on my third battery on the G4.  They routinely last about a year.  No big deal.  Go online order a new one when it gets a little weak.  Have a spare. Take it on a trip.  But to have to take the machine into Apple and get it replaced.  Don't know the details yet, but it sounds like I am without a laptop for a few days.  Not acceptable.

So here I sit.  With money to buy a new machine.  A little disappointed.  Not knowing what to do.  The Air is a lacking in very important features.  The Pro is a larger form factor than a prefer.

So that begs the question Air or Pro?  What would you do? 
 

iTarded Cubed

Aug 30, 2007 in Computing, Customer Focus

2

I got this note from Apple today:

Dear Lance,
Thank you for being a member of .Mac! Your .Mac membership is set to renew on September 27, 2007 PDT. Your credit card will be charged the day before your .Mac membership anniversary date and your account will renew for another year.

Please take a minute to review your account settings. If you want to change any of the details regarding your account, click here to update your Renewal Settings first.

I want to cancel my account but can not do so via the "click here" link. iTarded.

Why do I want to cancel. "Because it is not possible to simply change your .Mac member name." iTarded.

"If you activated or renewed your .Mac account within the past 30 days through the .Mac website with a credit card, you can cancel your .Mac account for a prorated refund." So what exactly I do when you send me an email with no option to cancel? Let them bill me and then cancel? iTarded.

There is a big big difference between being a hardware company and a service provider. Not sure if Apple gets that just yet.