Google’s Palimpsest project: promiscuous distribution of all science data sets
Posted by attilachordash on September 25, 2007
Google’s Palimpsest project, once realized (in the near future) has the potential to change the way science is done by accepting gigantic (raw?) data sets from all disciplines and making them open and free (including dark data?). Jon Trowbridge from Google Inc. had a presentation on SciFoo, 2007 at the Googleplex not documented well, but you can download his slides on the project that was presented at XTech 2007 in Paris, this May: Making Massive Datasets Universally Accessible and Useful Presentation. You are not restricted to the zip file as Jon kindly gave a permission to publish his slides with SlideShare here. From his intro: This talk will discuss a project underway at Google to collect and distribute large scientific datasets using a 21st century “Sneakernet”: multi-terabyte disk arrays shipped via FedEx and other common carriers.
The project is strictly non-profit, but fits well with Google’s mission.
Other links:
Deepak said
Attila
This is great. Thanks for posting.
» Google and promiscuous distribution of data » business|bytes|genes|molecules said
[...] Google about Google’s efforts in this direction. While the talk from Scifoo is not available, Attila got permission to upload Jon’s talk up on Slideshare. The presentation is quite similar to the talk at [...]
Pedro Beltrao said
Thanks for posting the slides. It is interesting. This is still very much for very large data volume but maybe whatever they build around this (maybe a GBase segment for scientific data) could be use for lower data volume uploaded via net.
Deepak said
Pedro,
At least for now they don’t necessarily have plans to do much with the data other than make it available on the web. Ideally, I think a Freebase/GBase type approach would be great. With an appropriate API and knowledge of the data structure, people could start building apps and of course, Google would do a great job of indexing the whole thing
Dark Data « Patient Centric Healthcare said
[...] Google’s Palimpsest project [...]
Tech News » Blog Archive » Google to Host Terabytes of Open-Source Science Data said
[...] (12:01pm): Attila Csordas of Pimm has a lot more details on the project, including a set of slides that Jon Trowbridge of Google gave at a presentation in [...]
Digging Digitally » More on Google: Free Hosting of Open Science Data said
[...] “Palimpset”. The Wired piece also links to this blog, “Pimm”, which has a presentation about this project available on Slideshare. Pimm’s blog said that this project is strictly nonprofit (I [...]
Google, gapminder and scientific data sets : business|bytes|genes|molecules said
[...] For more on Google and it’s data efforts, keep tabs on Attila’s blog, included his post on the promiscuous distribution of large datasets [...]
Google Said To Be Prepping Launch Of Social Science Data Network said
[...] annotation and commentary solution. What does that mean, exactly? Heck if I know. Venture over to this Pimm blog post to cycle through a brief slide show to get some measure of what one will likely encounter on launch [...]
Steffen Prohaska » Blog Archive » Storing Scientific Data at Google? said
[...] data on http://research.google.com/. Alexis Madrigal mentions more details in a post on Wired. Pimm’s post on the same topic displays the device to send data on slide 10/16. Google seems to plan to collect [...]
Google to host Open Source scientific data sets at The Musings of Chris Samuel said
[...] (including about why Google intend to import data by shipping RAID arrays around the world) here and (more up to date) [...]
Simone Cortesi » Blog Archive » Google Research said
[...] Il progetto sarà accessibile a tutti gli scienziati che desiderano condividere con la comunità i propri dati, vista la mole di dati da trasportare, il pogetto sfrutterà il vecchio adagio informatico: “niente ha la larghezza di banda di un TIR che viaggi in autostrada carico di hard disk”, e così faranno: gli scienziati riceveranno una valigetta contentente un black box sul quale caricare 3TB di dati che poi saranno fisicamente spediti a Googl per l’inclusione nel grande database. Il dataset con il quale il progetto partirà è quello delle fotografie dell’Hubble Space Telescope. Maggiori dettagli e alcune slide sull’aromento disponibili qui. [...]
The Fluff and the Mediocrity » Science News Round-Up #2 said
[...] Wired is reporting that Google will begin hosting terabytes of open-source data at http://research.google.com(the project was supposed to open this week, but missed the deadline, but will debut soon). The space will be free to scientists and the data will be available to all. The project is known as Palimpsest. The storage will allow scientists to explore amazing amounts of data. Tons of more information on the project is available at Pimm. [...]
Il blog dei Marsiaj - » Google ospitarà terabytes di dati scientifici open-source ? said
[...] Csordas di Pimm ha maggiori dettagli sul progetto, compresa una serie di diapositive, che Jon Trowbridge di Google ha mostrato ad una presentazione a [...]
Google Decides to Host a Whole Lot of Scientific Data - Palimpsest Project said
[...] [via Wired and Pimm] [...]
Google lanzará una plataforma de servicios para cientificos - Blog de Dr. Max Glaser said
[...] y discutir sus estudios y datos con otros científicos del mundo. El proyecto tiene el nombre de palimpsest, que será manejado desde el dominio [...]
Communications said
Google is playing to win in the 700 MHz auctions
Many say Google will bid to lose in the upcoming 700 MHz auctions and many more are equivocating. The idea is Google’s entry alone will induce enough openness, and besides they couldn’t afford to become an operator. This shows a
Laser said
That’s great.
I am curious what browse/search feature Google will provide. It will be nice the data be well annotated using semantic web technology.
Google schenkt Forschern Speicherplatz « Thalex said
[...] ” Wired” meldet, hätte das Projekt mit dem Namen Palimpsest eigentlich schon letzte Woche online gehen sollen. Doch der Start musste verschoben werden, soll [...]
Lee Watkins said
We at the Ctr. for Inherited Disease Research routinely ship data from genome scans to PIs and back-and-forth to NLM/NCBI on large encrypted disk arrays. We also continually archive and will eventually have to delete all the level 0 or “raw” data – the actual image data from which the genomic data is derived. I think someday we will regret deleting this data since better algorithms are developed every day yet many of these studies use the very last of available DNA from a given research subject who may be dead or otherwise no longer available to extract more. Having someplace to store them for future re-analysis, imho, be a great service.
Now, what about the data from extremely high-res 3D scana of the world’s entire collection of several hundred thousand cuneiform tablets, the world’s oldest written records and in many ways the foundational documents of human civilization? It might be a few petabytes or so: http://www.jhu.edu/digitalhammurabi/
Rumors suggest Google is set to open scientific data store « The “Meta” Internet: The genesis of a “virtual” Silicon Valleys leveraging the power of the Internet. said
[...] method works well for all large datasets. A presentation by Jon Trowbridge at SciFoo (slides available here) makes a compelling argument that disk hardware capacity has consistently outpaced network [...]
Google向科学家提供免费数据储存 | 七平米 said
[...] Google透露了一个新的针对科学社区的项目：Palimpsest，网址research.google.com，用于储存数以TB计（一开始是3TB，可能会进一步扩充到20T）的开放性科学数据。科学家可免费储存数据，任何人都可以自由访问，网站已经在去年8月被科学家预先测试过。Palimpsest将以Google去年收购的数据分析公司Trendalyzer研究的数据可视化技术为基础，加上自己开发的信息检查和查询算法。新网站将提供YouTube风格的注释和评论功能。这里有一位参与测试的生物学家在去年写下的评述及幻灯片。 [...]
Google Shutters Its Science Data Service — instantwebmeetings.com - Video Collaboration, E Learning, Video Meeting, Unified Communications said
[...] by Google engineer Jon Trowbridge at SciFoo 2007 — the slides from a later version of the talk is archived on the Partial Immortalization blog — the project was going to store, for free, some of the world’s largest scientific [...]
Open Source Science : Scriptyx.com said
[...] Et des informations complémentaires ici January 20, 2008 | Filed Under Google, Open Source, OpenAcademic, hosting, research, science, [...]