Thursday, June 11, 2009

clouds and peer-to-peer

We've been asked a few times about the relationship between clouds and peer-to-peer systems, and we wanted to take this opportunity to respond.

Definitions
We differentiate between peer-to-peer (p2p) techniques and p2p systems. The former refers to a set of techniques for building self-organizing distributed systems. These techniques are often useful in building datacenter-scale applications, including datacenter-scale applications that are hosted in the cloud. For instance, Amazon's Dynamo datastore relies on a structured peer-to-peer overlay, as do several other key-value stores.

People often use "P2P" to refer to systems that use these techniques to organize large numbers of cooperating end hosts (peers) such as personal computers and settop boxes. In these systems, most peers necessarily communicate using the Internet, rather than a local area network (LAN). To date, the most successful peer-to-peer applications have been file sharing (e.g., Napster, BitTorrent, eDonkey), communication (Skype). and embarrassing parallel computations, such as SETI@home and BOINC projects.

Limitations
The main appeal of p2p systems is that their resources are often "free", coming from individuals which volunteer their machines' CPUs, storage, and bandwidth. Offsetting this, we see two key limitations of P2P systems.

First, p2p systems lack a centralized administrative entity that owns and controls the peer resources. This makes it hard to ensure high levels of availability and performance. Users are free to disable the peer-to-peer application or reboot their machine, so a great degree of redundancy is required. This makes p2p systems a poor fit for applications requiring reliability, such as web hosting, or other sorts of server applications.

This decentralized control also limits trust. Users can inspect the memory and storage of a running application, meaning that applications cannot safely store confidential information unencrypted on peers. Nor can the application developer count on any particular quantity of resources being dedicated on a machine, or on any particular reliability of storage. These obstacles have made it difficult to monetize p2p services. It should come as no surprise that, so far, the most successful p2p applications have been free, with Skype being a notable exception.

Second, the connectivity between any two peers in the wide area is two or three order of magnitude lower than between two nodes in a datacenter. Residential connectivity in US is typically 1Mbps or less, while in a datacenter a node can often push up to 1Gbps. This makes p2p systems inappropriate for data intensive applications (e.g., data mining, indexing, search), which accounts for a large chunk of the workload in today's datacenters.

Opportunities
Recently, there have been promising efforts to address some of the limitations of p2p systems by building hybrid systems. The most popular examples are data delivery systems, such as Pando and Abcast, where p2p systems are complemented by traditional Content Distribution Systems (CDNs). CDNs are used to ensure availability and performance when the data is not found at peers, or/and peers do not have enough aggregate bandwidth to sustain the demand.

In another development, cable operators and video distributors have started to test with turning the set top boxes into peers. The advantage of settop boxes is that, unlike personal computers, they are always on, and they can be much easily managed remotely. Examples in this category are Vudu, and the European NanoDataCenter effort. However, to date, the applications of choice in the context of these efforts have still remained file sharing and video delivery.

Datacenter clouds and p2p systems are not a substitute for each other. Widely distributed peers may have more aggregate resources, but they lack the reliability and high interconnection bandwidth offered by datacenters. As a result, cloud-hosting and p2p systems complement each other. We expect that in the future more and more applications will span both the cloud and the edge. Examples of such applications are:

  • Data and video delivery. For highly popular content, p2p distribution can eliminate the network bottlenecks by pushing the distribution at the edge. As an example, consider a live event such as the presidential inauguration. With traditional CDNs, every viewer on a local area network would receive an independent stream, which could lead to choking the incoming link. With p2p, only one viewer on the network needs to receive the stream; the stream can be then redistributed to other viewers using p2p techniques.
  • Distributed applications that require a high level of interactivity, such as massive multi player games, video conferences, and IP telephony. To minimize latency, in these applications peers communicate with each other directly, rather than through a central server.
  • Applications that request massive computation per user, such as video editing and real-time translation. Such applications may take advantage of the vast amount of computation resources of the user's machine. Today, virtually every notebook and personal computer has a multi-core processor which are mostly unused. Proposals, such as Google's Native Client aim to tap into these resources.

98 comments:

  1. There was an interesting talk at the HotCloud workshop yesterday on bridging these two ideas to make "volunteer clouds".

    Nebulas: Using Distributed Voluntary Resources to Build Clouds, Abhishek Chandra and Jon Weissman

    Some other interesting talks:
    Hot Cloud 09

    ReplyDelete
    Replies
    1. Thanks combination of suitable and useful information and well-written sentences that will certainly entice your sense.There are so multiple comments here that are really entertaining and conducive to me thanks for sharing a link especially for sharing this blog. Judi Bola

      Delete
  2. Identity management is also a potential significant application for P2P.

    ReplyDelete
    Replies
    1. Thanks for the cool information shared on cloud computing. I am gathering information about it.Useful information has been given about the cloud computing. Thanks for the information.bola tangkas asia, 338a casino

      Delete
  3. In a P2P world a user retains control of their credentials and provides these directly to others as required.

    ReplyDelete
  4. Didn't know about the limitiations of residential connectivity. Interesting stuff.

    ReplyDelete
    Replies
    1. An interesting discussion is worth comment. I think that you should write more on this topic, it might not be a taboo subject but generally people are not enough to speak on such topics. To the next. 338a terbaik

      Delete
  5. I'm curious what the poster meant by "embarrassing parallel computations, such as SETI@home and BOINC projects." They seem successful.

    ReplyDelete
  6. Interesting insights.

    Are those residential comms numbers real? Do they arise because of the distribution of performance (median << mean) or from asynchronous characteristics, or is the US just way behind the rest of the world in residential comms performance?

    Also, surely if the main constraint is bandwidth between peers over the interent, then the p2p apps will tend to migrate to the higher bandwidth hosts?

    ReplyDelete
    Replies
    1. This is the perfect blog for anyone who wants to know about this topic. You know so much its almost hard to argue with you (not that I really would want...HaHa). You definitely put a new spin on a subject thats been written about for years. Great stuff, just great!
      http://acmbet.net/bola-tangkas-baru/, http://arenaliga.com/338a-terbaik/

      Delete
  7. Interesting text. You have a nice blog. Keep it up!

    ReplyDelete
  8. This is a very detailed informational and educational post... I learned a very great deal from this peace... Sometime in the future, the clouds(location somewhere) will be where some parts of our lives are stored...

    __
    ParaSlim
    Acai Optimum

    ReplyDelete
  9. Great information provided. Thanks for that. By the way i had attended a conference previously. Its a Cloud Computing Virtual Conference 2009,Which is the global cloud computing event, covering latest trends and innovations in the world of cloud computing. It is the Annual Virtual Conference on Cloud Computing! It is hosted online March 23-25, 2009.

    ReplyDelete
  10. good~ keep sharing with us, please....I will waiting your up date everyday!! Have a nice day........................................

    ReplyDelete
  11. Excellent Information Shared. Cloud Computing is actually a great thing. I like to learn much about it. It is very important for my field, thats why I have planned to attend the Computing Conference which is going to be hosted online march 2010.

    ReplyDelete
  12. 發表,是一種抒發:好的心情可以一再的回味發酵;壞的心情就讓它留在文字裡吧!加油!........................................

    ReplyDelete
  13. 當一個人內心能容納兩樣相互衝突的東西,這個人便開始變得有價值了。......................................................

    ReplyDelete
  14. Nice blog,keep sharing with us, I look forward to your content

    www.chillyz.com

    ReplyDelete
  15. File sharing has its legitimate use as well. For example, media students need to transfer and share large files all the time. That is why private P2P based file sharing solutions for legitimate purposes, such as Binfer are gaining popularily. Binfer allows file sharing only with trusted friends and is not based on the torrent protocol. Check it out: http://www.binfer.com

    ReplyDelete
  16. 您的部落格文章真棒!!有空我一定會常來逛!!........................................

    ReplyDelete
  17. 好的開始並不代表會成功,壞的開始並不代表是失敗...............................................................

    ReplyDelete
  18. The connectivity between any two peers in the wide area is two or three order of magnitude lower than between two nodes in a datacenter.
    operatore video

    ReplyDelete
  19. 讚啦!!. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

    ReplyDelete
  20. 書籍是全世界的營養品,生活裡沒有書籍,就好像沒有陽光......................................................

    ReplyDelete
  21. Really a very great information shared over here. By the way have you heard about Cloud Computing Virtual Conference 2009. It is the event covering latest trends and innovations and technologies of the Cloud Computing. Cloudslam09 conference is the 1st annual conference on cloud computing.

    ReplyDelete
  22. 人應該做自己認為對的事,而不是一味跟著群眾的建議走。.......................................................

    ReplyDelete
  23. Blogs are so informative where we get lots of information on any topic. Nice job keep it up!!
    _____________________________

    Dissertation Proposal

    ReplyDelete
  24. 卡爾.桑得柏:「除非先有夢,否則一切皆不成。」共勉!............................................................

    ReplyDelete
  25. Easy Earning money in online never been this easy and transparent. You would find great tips on how to make that dream amount every Day.Sitting in the home earn around $100 perday. So go ahead and click here for more details and open floodgates to your online income. All the best.

    Thanks.

    ReplyDelete
  26. This comment has been removed by the author.

    ReplyDelete
  27. Nice blog writing the article on this blog article informative people get lots of

    interesting info from there keep it updated.

    Dissertation writing | Dissertation topics |

    Dissertation

    proposal

    ReplyDelete
  28. 若有人問你成功時會不會記得他 試問若你失敗時他會不會記得你......................................................................

    ReplyDelete
  29. 在莫非定律中有項笨蛋定律:「一個組織中的笨蛋,恆大於等於三分之二。」............................................................

    ReplyDelete
  30. At the moment I am trying to learn the concept of cloud computing. Your article has provided me with a nice starting insight. I have found this site offering a free trial of services such as FTP server hosting, file sharing plug-ins for blogs, and cloud file storage. You may want to review their services. I, however, will give it a try in order for me to get a stronger and solid grasp of this wonderful concept of cloud computing.

    ReplyDelete
  31. Thanks for sharing.

    Tenax Technologies is a Belarussian software development company delivering complex web solutions. We provide comprehensive Java development of complex internet systems.
    software company

    ReplyDelete
  32. Thank you for such a fantastic blog. Where else could anyone get that kind of info written in such a perfect way? I have a presentation that I am presently working on, and I have been on the look out for such information.
    Antique Shopping Online

    ReplyDelete
  33. Wonderful article,thanks for putting this together! "This is obviously one great post. Thanks for the valuable information and insights you have so provided here. Keep it up!"
    Dissertation help

    ReplyDelete
  34. This comment has been removed by the author.

    ReplyDelete
  35. just wanna share that a good web hosting is a matter >>> javahostindo web hosting indonesia

    ReplyDelete
  36. In this era of blog ,we easily get nice & updated information for research purposes... I'd definitely appreciate the work of the said blog owner... Thanks!
    seo agency sydney

    ReplyDelete
  37. Excellent instruction this blog gives me and Well differentiate the blog owner between peer-to-peer (p2p) techniques and p2p systems. The former refers to a set of techniques for building self-organizing distributed systems. These techniques are often useful in building datacenter-scale applications, including datacenter-scale applications that are hosted in the cloud.
    Saint-Petersburg Hotels

    ReplyDelete
  38. Apple’s next generation handsets i.e. iphone 4S and iphone 5 will hit the stores this year....The Best Android

    ReplyDelete
  39. This is really great and informational blog with brilliant posts.

    ReplyDelete
  40. Centro Metro This is the perfect blog for anyone who wants to know about this topic. You know so much its almost hard to argue with you (not that I really would want...HaHa). You definitely put a new spin on a subject thats been written about for years. Great stuff, just great!

    ReplyDelete
  41. Such a wonderful post. I got useful info from this post. keep it up and thanks for sharing. :)

    3d ultrasounds | 4d ultrasounds

    ReplyDelete
  42. This is an excellent post I seen thanks to share it. It is really what I wanted to see hope in future you will continue for sharing such a excellent post. ~^- organic supplements -^~

    ReplyDelete
  43. This is such a terrific and useful topic that you’re shared with. I’m satisfied by the way you brought out ideas. Thanks for sharing.
    Best Graphic Design Site

    ReplyDelete
  44. asdadsad sandbnasbdbn
    asdjashjdk sadsadnmnm asnmdm sadkaskd asdasdkskad kata mutiara bahasa inggris dan zodiak terbaru hari ini paling mantab
    Kata Mutiara
    Zodiak Hari Ini
    sdadnhjkqwqw

    ReplyDelete
  45. It is my pleasure that I have the unique opportunity to comment on this awesome post.

    find doctors list | doctor ratings

    ReplyDelete
  46. Interesting post. I have been wondering about this issue, so thanks for posting.

    natural sleep aid | energy supplements | fast energy

    ReplyDelete
  47. Thank you for some other great article. The place else could anyone get that type of information in such a perfect approach of writing? I’ve a presentation subsequent week, and I’m at the search for such info.

    Sepeda Motor Injeksi Irit Harga Terbaik Cuma Honda :: Kata Mutiara :: Kata Bijak Persahabatan :: Kata Gokil :: Klasemen Liga Italia :: SMS Gokil :: Sepeda Motor Injeksi Irit Harga Terbaik Cuma Honda :: Zodiak :: Ramalan Jodoh :: Ramalan Cinta :: Ramalan Bintang :: Kata Mutiara :: Cerita Lucu :: Sepeda Motor Injeksi

    ReplyDelete
  48. I like your blog post. Keep on writing this type of great stuff. I'll make sure to follow up on your blog in the future.
    masters dissertation proposal

    ReplyDelete
  49. Thanks for posting this info. I just want to let you know that I just check out your site and I find it very interesting and informative. I can’t wait to read lots of your posts. Agen Judi Bola Sbobet Ibcbet

    ReplyDelete
  50. Thank you!, Lots of people have to be aware of the impact that’s being made towards the author of this blog.
    Marketing communication strategy | Advertising as profession in Estonia | Advertising professional in Estonia

    ReplyDelete
  51. To create a such kind of article is really amazing,I daily read your blogs and give my announcement for that here this article is too great and so entertaining. Judi Online

    ReplyDelete
  52. At TechTrendsIT we believe that Cloud Application Development Service will drive future technology strategy and help business grow effectively. We suggest and give a strong recommendation to our clients to consider a “cloud first policy” while designing and deploying web application and mobile applications.

    ReplyDelete
  53. Thanks for making the honest effort to discuss this. I feel very sturdy approximately it and wish to read more. If it’s OK, as you achieve more in depth knowledge, could you thoughts including extra articles very similar to this one with more information? It would be extremely useful and helpful for me and my friends. model rambut layer | gaya rambut wanita 2014 | model rambut wanita panjang lurus | cara membuat rambut tidak mengembang | model rambut wanita korea 2014 | model rambut cowok korea

    ReplyDelete
  54. This will likely almost certainly be rather beneficial for many of your job opportunities I want to you should not only with my blog site. Situs OLB365.COM AGEN JUDI BOLA ONLINE, AGEN JUDI CASINO ONLINE INDONESIA TERPERCAYA, Coba Indomasterbola.com Agen Judi Bola, Taruhan Bola Terbaik dan Terpercaya

    ReplyDelete
  55. Toko Alat Bantu Sex Meyediakan berbagai macam alat sex, yang dapat di manfaatkan untuk pria dan wanita dalam mencari kepuasan biologis.
    Alat Bantu Sex Wanita.
    Alat Bantu Sex pria.

    ReplyDelete