The TIME Warp Strategy News...

1. Arranging Things: The Rhetoric of Object Placement (Amazon) -- [...] the underlying principles that govern how Western designers arrange things in three-dimensional compositions. Inspired by Greek and Roman notions of rhetoric [...] Koren elucidates the elements of arranging rhetoric that all designers instinctively use in everything from floral compositions to interior decorating. (via Elaine Wherry)
2. 2010 Mario AI Championship -- three tracks: Gameplay, Learning, and Level Generation. Found via Ben Weber's account of his Level Generation entry. My submission utilizes a multi-pass approach to level generation in which the system iterates through the level several times, placing different types of objects during each pass. During each pass through the level, a subset of each object type has a specific probability of being added to the level. The result is a computationally efficient approach to generating a large space of randomized levels.
3. Wave in a Box -- Google to flesh out existing open source Wave client and server into full "Wave in a Box" app status.
4. 3D Sound in Google Earth (YouTube) -- wow. (via Planet In Action)

For an agency that is charged with regulating communications, the Federal Communication Commission (FCC) has been a bit behind the curve in evolving its online presence. FCC.gov was launched in June 1995, redesigned in 1999, and relaunched again in September 2001. Since then, it has remained a largely static repository for public notices and information about the agency's action.

According to FCC officials, that's going to change, and soon. There was already some insight offered into redesigning the FCC website back in January on the agency blog, informed at least in part by discussions with Sunlight Labs on redesigning the government.

Yesterday, I interviewed FCC managing director ?Steven VanRoekel at FCC headquarters about what rebooting FCC.gov? will mean for the agency, businesses and the American people. "The new site will embrace open government principles around communication and participation," said VanRoekel. "Consider OpenInternet.gov, where over 30,000 ideas were generated, or Broadband.gov. Comments there go into the official record and are uploaded to the Library of Congress. You will see that in a much more pervasive way in the new FCC.gov."

Our short video interview is below. An extended interview follows.

Redesigning FCC websites for public comment

In January, the FCC launched Reboot.gov and asked for public input on improving citizen

Note: Data Week is a new series that collects notable stories and developments from the data world.

Data visualization and art

Artist and scientist Stephen von Worley announced the launch of a new blog, Data Pointed, showcasing his data visualization research. "One part magazine and two parts blog", the site tells the story of von Worley's own data visualizations, and surveys choice picks from others.

The lead story covers the results of the XKCD color name survey, illustrating the entertaining differences in the way different genders refer to colors.

Across the top, witness the nuanced verbal repertoire of feminine color differentiation. While us men are busy grunting, guzzling beer, and shoving our hands down our pants, women get specific by mixing fruits, animals, spices, flowers, and other such familiarities with finely-honed modifiers like neon and dusty. The result? A vast panoply of warm-fuzzy color names that seemingly trounces anything our Y-chromosomes have to offer.

The visualization shows colors organized horizontally by hue, and vertically by gender preference. Immediately obvious is the contrast in nuance between female, at the top, and male. Click the graphic below to reach the full article and interactive visualization.

Becoming a data scientist

The community Q&A site Quora is rich with information about data science, analytics and computing. An especially illuminating answer was given this week to the question How do I become a data scientist — how does someone with a computer science background get the math and statistics knowledge required for data science?

Providing an extensive reply, Alex Kamil gives eight points from his perspective as an undergraduate student. Many of these reference statistics and math, and Kamil provides an excellent list of papers, websites and technologies to tinker with.

Several of Kamil's suggested starting points struck me as common themes among those who define themselves as data scientists:

• Start learning statistics by coding with R: whatever the size of the data you're working with, many data analysts perform and prototype investigations using the R language. Some will later translate these into larger map-reduce jobs to be run on Hadoop, for instance. R provides a hands-on way for developers to teach themselves statistics in practice.
• Linear algebra: a grounding in linear algebra is common among many data scientists, and important because matrix math underpins many data mining applications, such as the famous PageRank.
• Machine learning: allowing computers to alter behavior based on input data is fundamental to many innovative data-based products and services. Many developers start this ad-hoc, but there is much available literature. Kamil references Bradford Cross' extensive list of machine learning resources.

There are many more starting points referenced in the full answer.

The field of data science is a place where book learning meets code and produces results. In the words of Kurt Lewin: "There's nothing so practical as a good theory."

CouchDB clusters and particles

Cloudant, providers of hosted CouchDB infrastructure, have released their clustering technology, BigCouch, as open source . CouchDB is a document-oriented "NoSQL" database, noted for its replication features. Use it as part of your application and you can count on database replication "for free."

As part of offering cloud-based CouchDB services, Cloudant has developed software to create clusters of CouchDB instances, distributed among many servers. In Cloudant's words, "Instead of one big honking CouchDB, the result is an elastic data store which is fully CouchDB API-compliant."

The most direct comparison to existing technologies is with Amazon's Dynamo, according to the announcement:

The clustering layer [features] consistent hashing, replication, and quorum for read/write operations. CouchDB view indexing occurs in parallel on each partition, and can achieve impressive speedups as compared to standalone serial indexing.

You can download BigCouch from its Github Project Page, and collaborate on the #cloudant IRC channel on Freenode.

Elsewhere this week, ReadWriteEnterprise reports that scientists working at CERN are using CouchDB to support their work on the Compact Muon Solenoid Experiment (CMS) on the famous Large Hadron Collider.

Chief among the attractions of CouchDB to the scientists is the ability to manage petabytes of data, the built-in replication features, and an easy compatibility with Oracle. Simon Metson of the Data Management and Workflow Management group at the CMS project reports that CouchDB has a shallow learning curve, but is harder for those with deep SQL backgrounds: "The more you know Oracle, the harder it is to pick up."

Strata: The Business of Data

If you enjoyed any of the previous items, stay tuned — we're excited to announce the launch of Strata, an O'Reilly conference focusing on the business and practice of data. The conference will be held in Santa Clara, Calif. from Feb. 1-3, 2011.

At O'Reilly, we believe that the future belongs to those who understand how to collect and use their data successfully. There's a change in both the skills of data analysts and the technology they use that's sweeping through industry and science. Our aim with Strata is to be the defining event for that change: for practitioners, businesses and data vendors.

The call for participation is open until Sept. 28. We're looking for proposals from practitioners, business leaders, analysts, designers, and developers covering the spectrum of data business and practice. Suggested topics include:

• Distributed data processing, Hadoop ecosystem
• From research to product
• Streaming data processing
• Becoming a data-driven organization
• Data science best practices
• Data acquisition, cleaning, distribution and markets
• Machine learning
• Training and recruitment of data scientists
• Applications, case studies, and cautionary tales
• Visualization and design principles
• Augmented reality and immersive interfaces
• Data protection, privacy, and policy
• Changing role of business intelligence

Send us news

Email us news, tips and interesting tidbits at dataweek@oreilly.com.

Three significant trends in the local sector -- deals, check-ins, and place pages -- are on a bender and headed for an exciting convergence. When they meet we will see one of three things: a train wreck of incompatibility, an awkward confluence, or a very powerful alignment. I'm hoping for the latter, a sort of local syzygy, because a well-conceived orchestration of these trends will benefit the consumer and it has real potential to take us entirely out of the Yellow Pages era and into exciting, unexplored territory.

This is a two-part post: here I look in more detail at check-ins, deals, and place products (including, briefly, the adventurously named Facebook Places) with an eye to what might follow. In a following post I will discuss how we may more actively ease their convergence with linked data and some basic adherence to extant standards, specifically how these efforts will affect the local consumer.

Place pages and check-ins

The check-in is hardly the apogee of the local consumer experience but it works, and this is what is most important about any product. However successful it is now, the check-in will remain an interim solution for identifying long-term customer/business affinities and physical point of presence. So what's next?

I've written about check-ins previously: since then, Facebook has thrown its hat into the ring with their own place/check-in product, offering little feature distinction outside the problematic ability to check-in your friends on their behalf . Thanks to Facebook, the "Ferris Bueller Problem" -- in which a friend checks you into (say) the Von Steuben Day Parade when you are officially at home, ill -- may soon find its way into mainstream parlance. Expect a rise in just-for-larks in absentia check-ins to the local "gentlemen's club" and similar places of sophomoric amusement.

More interestingly, casual requests for a similar product from LinkedIn, and the introduction of a third-party check-in offering for Twitter demonstrate that geo and social products are becoming more integrated in the mind of the consumer, and corporate product strategies: Greg Sterling remarked on this trend recently in yp.com's new eat, play, live marketing campaign which attempts to transform the brand from its staid origins into "a lifestyle guide that also happens to feature contractors and plumbers." He is spot-on: some lines of business will not fit within the check-in model, but they nonetheless must be accommodated in any successful business-to-consumer product.

I have no desire to see another check-in clone arrive anytime soon. Jeff Holden, the founder of Whrrl, recently noted that the check-in will shortly be a commodity, and Foursquare's Dennis Crowley believes it already is. If you are an entrepreneur or developer thinking of building a new check-in service, please don't. Instead, consider some of the more exciting challenges that provide real consumer benefit:

• Divorce the business from user authentication and social network: Social/local apps are entirely insular: you must be a Foursquare user using a Foursquare app (or the Foursquare API) to check into a Foursquare venue. I can see why this is, but it need not be. For example, consider an alternative platform where users supply instead an authenticated OpenID and an hcard. Authenticated user-to-venue relationships could be exposed across multiple social networks under the user's control, and the access control list could even accommodate attribute criteria such as place type or business category. This is not simple aggregation or shimming, but a lower-level, cross-platform integration.
• Auto check-in: Manual check-ins are so 2009. Instead, think toward the endpoint where a geo-commercial footprint will be created automatically as an artifact of our day-to-day activities, should we wish it. The first steps in this direction will be toward improving the consumer's retail experience. Shopkick is effectively an auto check-in service. Other efforts toward this end will include credit-card geo traces, perhaps combined with a "local favorites" list used to prescribe which businesses may be checked into automatically. There are critical privacy implications here, so this footprint must be controlled by users, and employed by small and medium businesses (SMBs) to engage their customers on their own terms -- offering deals and events rather than advertising. As sharing your location with others becomes more codified, it will place demands on forthcoming products and start-ups that expose this under very granular, controlled conditions. This includes timestamps, geofences, disparate social networks, and leveraged venue classification.
• Micro beacons: Expect over the next two years a bloom of micro-positioning products: sensor-and-platform combinations that assist with indoor navigation (malls), determining when a user is genuinely "in store," and guiding the consumer directly to a product on-shelf. Shopkick's iPhone product uses the device's microphone (simple, clever) to do this, but "Bluetooth beaconing" and QR codes are other low-energy, low-hassle options. We shall see both passive and active implementations to assist with the auto check-in and location disambiguation, depending on the particular use case.

Advances along these tracks should obviate the check-in as we know it today. This is a good thing -- check-ins are something to get over, an intermediate solution to tolerate until we have something that works better. Foursquare certainly knows this. The excitement -- for Foursquare's business and users -- lies wholly in their ability to deliver utility, novelty, and serendipity beyond the check-in.

Get it here: the deal

Groupon's $134 million series C funding in April and its estimated $1.34 billion valuation woke investors and entrepreneurs to the monetary value of group buying, and local deals more generally. There are now hundreds of variants and multiple aggregators, while "Groupon clone scripts" can be purchased (caveat emptor) from any number of freelance developer sites.

Groupon and its ilk tend to get bundled under the "group buying" or less-apt "social couponing" monikers, but -- in regards to Groupon certainly -- there's little that's actively social about the products. The less charitable may argue that Groupon's success is due as much to the severity of discounts on offer. However, Groupon and others have raised awareness that advertising is no longer the only solution. Specifically, SMBs are slowly gaining access to tools to engage their customers on mutually favorable terms. Examples:

• Just-in-time delivery: Businesses increasingly have the ability to deliver discounts, incentives, and upsells to consumers in a highly relevant, just-in-time manner. Think of services that provide attractive discounts when tables are open at a restaurant, when spare inventory is available, or the business simply requires immediate revenue to help with cashflow. These new approaches secure short-term, incremental revenue for the business and offer great value to the user. Win-win.
• Topical content: Most of these products now deliver deals based on city and zip code with little accommodation for either personal preferences or current location. Expect "relevance" to become the new watchword in deal and coupon delivery. Relevance will be obtained via platforms that exploit an accessible selection of local business favorites: Facebook is in a position to deliver, as is Hunch local, or my former shop, LikeList.
• Coupon organizers: Digital deals are big, and will become only more common as delivery channels are improved. Jim Moran, co-founder of daily deal aggregator Yipit, records more than 100 companies offering deals in the U.S. alone, most recently Yelp. With these numbers, digital deals threaten to become as unmanageable as their hardcopy equivalents. In the same way that local favorites will assist in filtering the signal from the noise in the forthcoming local clamor, I would expect that intelligent, digital coupon wallets will be employed to remind or introduce users to deals that are relevant to their immediate location and circumstance. One coupon vendor called Savings Sidekick (apparently not related to the Shopkick iphone app) is doing something similar by using the tip feature in Foursquare to remind users of nearby deals from their coupon books. This particular implementation is as interesting as it could be annoying.

Part 1 wrap

Local is huge and only getting bigger. As a litmus, Borrell's recent ad forecast notes that "local online advertising should grow by almost 18% [...] to $16.1 billion, in 2011." Money follows money: we can expect further me-too products around deals, check-ins, and place products, but there is huge scope for investment into products that contribute genuine value to the consumer experience and enhance SMBs' ability to connect to their customers.

Much of this will take place at the data and platform levels. In my next post I'll take a look at how linked data might help cross-platform integration, and join deals, check-ins and place pages to the benefit of the consumer.

Related:

• Why check-ins and like buttons will change the local landscape
• Where's the map?
• Foursquare wants to be the mayor of location apps
• What you need to know about local search

Note: Data Week is a new series that collects notable stories and developments from the data world.

Data visualization and art

Artist and scientist Stephen von Worley announced the launch of a new blog, Data Pointed, showcasing his data visualization research. "One part magazine and two parts blog", the site tells the story of von Worley's own data visualizations, and surveys choice picks from others.

The lead story covers the results of the XKCD color name survey, illustrating the entertaining differences in the way different genders refer to colors.

Across the top, witness the nuanced verbal repertoire of feminine color differentiation. While us men are busy grunting, guzzling beer, and shoving our hands down our pants, women get specific by mixing fruits, animals, spices, flowers, and other such familiarities with finely-honed modifiers like neon and dusty. The result? A vast panoply of warm-fuzzy color names that seemingly trounces anything our Y-chromosomes have to offer.

The visualization shows colors organized horizontally by hue, and vertically by gender preference. Immediately obvious is the contrast in nuance between female, at the top, and male. Click the graphic below to reach the full article and interactive visualization.

Becoming a data scientist

The community Q&A site Quora is rich with information about data science, analytics and computing. An especially illuminating answer was given this week to the question How do I become a data scientist — how does someone with a computer science background get the math and statistics knowledge required for data science?

Providing an extensive reply, Alex Kamil gives eight points from his perspective as an undergraduate student. Many of these reference statistics and math, and Kamil provides an excellent list of papers, websites and technologies to tinker with.

Several of Kamil's suggested starting points struck me as common themes among those who define themselves as data scientists:

• Start learning statistics by coding with R: whatever the size of the data you're working with, many data analysts perform and prototype investigations using the R language. Some will later translate these into larger map-reduce jobs to be run on Hadoop, for instance. R provides a hands-on way for developers to teach themselves statistics in practice.
• Linear algebra: a grounding in linear algebra is common among many data scientists, and important because matrix math underpins many data mining applications, such as the famous PageRank.
• Machine learning: allowing computers to alter behavior based on input data is fundamental to many innovative data-based products and services. Many developers start this ad-hoc, but there is much available literature. Kamil references Bradford Cross' extensive list of machine learning resources.

There are many more starting points referenced in the full answer.

The field of data science is a place where book learning meets code and produces results. In the words of Kurt Lewin: "There's nothing so practical as a good theory."

CouchDB clusters and particles

Cloudant, providers of hosted CouchDB infrastructure, have released their clustering technology, BigCouch, as open source . CouchDB is a document-oriented "NoSQL" database, noted for its replication features. Use it as part of your application and you can count on database replication "for free."

As part of offering cloud-based CouchDB services, Cloudant has developed software to create clusters of CouchDB instances, distributed among many servers. In Cloudant's words, "Instead of one big honking CouchDB, the result is an elastic data store which is fully CouchDB API-compliant."

The most direct comparison to existing technologies is with Amazon's Dynamo, according to the announcement:

The clustering layer [features] consistent hashing, replication, and quorum for read/write operations. CouchDB view indexing occurs in parallel on each partition, and can achieve impressive speedups as compared to standalone serial indexing.

You can download BigCouch from its Github Project Page, and collaborate on the #cloudant IRC channel on Freenode.

Elsewhere this week, ReadWriteEnterprise reports that scientists working at CERN are using CouchDB to support their work on the Compact Muon Solenoid Experiment (CMS) on the famous Large Hadron Collider.

Chief among the attractions of CloudDB to the scientists is the ability to manage petabytes of data, the built-in replication features, and an easy compatibility with Oracle. Simon Metson of the Data Management and Workflow Management group at the CMS project reports that CouchDB has a shallow learning curve, but is harder for those with deep SQL backgrounds: "The more you know Oracle, the harder it is to pick up."

Strata: The Business of Data

If you enjoyed any of the previous items, stay tuned — we're excited to announce the launch of Strata, an O'Reilly conference focusing on the business and practice of data. The conference will be held in Santa Clara, Calif. from Feb. 1-3, 2011.

At O'Reilly, we believe that the future belongs to those who understand how to collect and use their data successfully. There's a change in both the skills of data analysts and the technology they use that's sweeping through industry and science. Our aim with Strata is to be the defining event for that change: for practitioners, businesses and data vendors.

The call for participation is open until Sept. 28. We're looking for proposals from practitioners, business leaders, analysts, designers, and developers covering the spectrum of data business and practice. Suggested topics include:

• Distributed data processing, Hadoop ecosystem
• From research to product
• Streaming data processing
• Becoming a data-driven organization
• Data science best practices
• Data acquisition, cleaning, distribution and markets
• Machine learning
• Training and recruitment of data scientists
• Applications, case studies, and cautionary tales
• Visualization and design principles
• Augmented reality and immersive interfaces
• Data protection, privacy, and policy
• Changing role of business intelligence

Send us news

Email us news, tips and interesting tidbits at dataweek@oreilly.com.

1. Guardian Science Blogs -- the latest in a series of science blog aggregators. Nobody is too sure what benefits a blog umbrella like Discovery or Nature (or the Guardian) offers bloggers. Regardless of this, the content is fantastic.
2. v2ex: A Community Running on AppEngine -- no hosting costs, massive scalability.
3. Raising Money for Vanuatu Arts Center -- a Kickstarter project to fund a 6-hectare/14.8-acre off-the-grid artists retreat, cultural preservation and technological education space in the remote Pacific island of Vanuatu. Kickstarter is incredible. (via BoingBoing)
4. Orbiter (XKCD) -- names are human artifacts, as every Internet mapping company knows. I'm reminded of how Gracenote, who run CDDB, store every datum submitted to them, and consequently have nearly fifty spellings of Britney Spears.

With the IFA Consumer Electronics Unlimited techno-smorgasbord set to open this Friday, there's a lot of buzz going around about upcoming announcements and unveilings. Much of the pre-show buzz is centered around Android-based competition for the Apple iPad.

The IFA traditionally offers an early indication of what gadgets will sell well through Christmas. With order volume stemming from last year's show reaching nearly $3.8 billion, it's no wonder so much attention is focused on the show .

Toshiba's SmartPad

Images of the new Toshiba SmartPad were recently leaked, but with little concrete information about the device. Some news sources are claiming that the new Android 2.2-powered device will actually be called the Toshiba Folio 100. There may also be a Windows 7 version of the SmartPad as well. Little is available from Toshiba itself other than the device is set to ship in October. This is one of the devices that is expected to be unveiled at IFA.

ViewSonic ViewPad 7

With a screen that measures 7-inches across, the ViewPad is jokingly being referred to as the world's largest phone. Featuring both a touchscreen tablet, the ViewPad also provides a slot for a full-sized SIM card that will offer both voice and 3G data. In addition, the ViewPad will feature a 3 megapixel front- and back-facing camera. The device will run Android 2.2. ViewSonic has not unveiled pricing or availability -- those details will probably be announced at IFA. However, industry analyst reports indicate that the device will be around $550 and launch sometime in October.

Archos Android Internet tablets

This week brought the announcement from Archos that they were launching five new Android tablets. The tablets include web-browsing, games, ebook, social networking, and HD video and music. Prices range from $99.99 to $349.99. The Archos 28 and 32 are slated for release this month. Models 43, 70, and 101 are scheduled for later this fall. [Editor's Note: Archos' product names include a smaller adjacent number.]

Sharper Image Literati

Once a common occupant in malls throughout the US, Sharper Image has fallen on tough times -- filing for Chapter 11 bankruptcy in February of 2008. While the only thing that remains of the company is their website, that hasn't prevented them from joining the ereader fray.

Resembling the Amazon Kindle, the Kobo-powered color ereader will launch at $160. Featuring a QWERTY keyboard below a 7-inch screen, the ereader lacks a web browser and application functionality, clearly putting it in the standalone ereader category. Expected to arrive in October, the Literati will be available from a number of retailers, including Best Buy, Bed, Bath & Beyond, JC Penney, Kohl's and Macy's.

Acer LumiRead

Trying to build upon their netbook success, Acer recently announced its LumiRead ereader device. Based on a 6-inch E Ink display, the LumiRead will feature a Kindle-style QWERTY keyboard below the screen. When combined with either the 3G or WI-FI models, the LumiRead's Digital Living Network Alliance (DLNA) compliance makes the ereader capable of streaming music from Acer's own clear.fi system and third-party streaming services. The LumiRead also includes a built-in web browser with a "Smart Download" feature, which allows the LumiRead to save local versions of web pages for reading later when the device is disconnected from the Internet.

The LumiRead will come with access to the Barnes & Nobles ebook store as well as Libri.de, Germany's leading Internet book retailer. While no prices have been announced, rumors suggest the LumiRead will start shipping in October for around $316.

SigmaTek eReaders

Coming in a 5-inch and a 7-inch ereader, the devices from Sigmatek Computer both feature a TFT display capable of a 800x480 resolution. These devices offer multimedia features that include support for AVI, XviD, and MKV video, with MP3, WMA, FLAC, AAC, WAV, and OGG music formats.

From the start, these ereaders may be playing catch-up because the devices are not equipped with connectivity to any application or ebook store. Some may see this as an advantage because it means the ereaders will be open to grab EPUB, PDF and TXT files from anywhere. In stores in October, the 5-inch model will cost approximately $127 and the 7-inch approximately $153.

Other announcements from the world of ereaders

Following a similar deal with retailer Target, Staples will start selling Amazon's Kindle in its stores this autumn. While expanding the number of channels for selling Kindle devices is key, according to Chris Brogan, Amazon's willingness to port their Kindle application to other ereader platforms may become their most important channel.

According to an SEC filing by electronics manufacturer LG, they could be mass-producing 9.7-inch color and 9-inch flexible e-paper displays by the end of the year. According to an analyst at Forrester, the availability of flexible screens could greatly improve the durability of ereaders. While flexible ereaders from Skiff and Plastic Logic have failed in the past due to heavy pricing competition, a mass-produced display from LG could level the pricing playing field and bring about new ereader innovations.

Finally, as part of a unique, year-long Notre Dame study of ereaders, the university debuted their first class taught entirely using the Apple iPad. The iPad will replace the textbook previously used in assistant professor Corey Angst's project management course. Members of the study are evaluating the iPad with the broader goal of designing an epublishing ecosystem