Comments on: MySQL: Data Storage or Data Processing http://www.mysqlperformanceblog.com/2007/11/18/mysql-data-storage-or-data-processing/ Everything about MySQL Performance Sat, 21 Nov 2009 05:23:57 -0800 http://wordpress.org/?v=2.8.4 hourly 1 By: Gaurav Singh http://www.mysqlperformanceblog.com/2007/11/18/mysql-data-storage-or-data-processing/comment-page-1/#comment-281705 Gaurav Singh Fri, 18 Apr 2008 19:31:27 +0000 http://www.mysqlperformanceblog.com/2007/11/18/mysql-data-storage-or-data-processing/#comment-281705 how i can switch mysql to use other partition of my hard disk to store data. i am a new kid for mysql. assist me for the same. how i can switch mysql to use other partition of my hard disk to store data.
i am a new kid for mysql. assist me for the same.

]]> By: peter http://www.mysqlperformanceblog.com/2007/11/18/mysql-data-storage-or-data-processing/comment-page-1/#comment-203609 peter Sat, 24 Nov 2007 19:54:35 +0000 http://www.mysqlperformanceblog.com/2007/11/18/mysql-data-storage-or-data-processing/#comment-203609 Well Google is just an example. It could be Amazon, FaceBook or Yahoo... a lot of smaller second tier web sites would qualify as well. Web system which have to deal with user generated content have to deal with massive amount of such. Speaking about Google - this is deliberate choice of search application. There are many services using same underlying infrastructure such as bigtable which do not have this problem. You do not really need to have shared storage to give consistent results, you can have consistent shared systems either. There are of course various implications - like if you get to deal with multiple data centers you have to choice either synchronous propagation (meaning you can run into the trouble if connection between them has problems) or asynchronous propagation, meaning you will get a stale copy. Of course you can structure the application so stale copy is never used but it normally comes with other performance challenges. Well Google is just an example. It could be Amazon, FaceBook or Yahoo… a lot of smaller second tier web sites would qualify as well.

Web system which have to deal with user generated content have to deal with massive amount of such.

Speaking about Google – this is deliberate choice of search application. There are many services using same underlying infrastructure such as bigtable which do not have this problem.

You do not really need to have shared storage to give consistent results, you can have consistent shared systems either. There are of course various implications – like if you get to deal with multiple data centers you have to choice either synchronous propagation (meaning you can run into the trouble if connection between them has problems) or asynchronous propagation, meaning you will get a stale copy. Of course you can structure the application so stale copy is never used but it normally comes with other performance challenges.

]]> By: Gints Plivna http://www.mysqlperformanceblog.com/2007/11/18/mysql-data-storage-or-data-processing/comment-page-1/#comment-201873 Gints Plivna Thu, 22 Nov 2007 16:07:02 +0000 http://www.mysqlperformanceblog.com/2007/11/18/mysql-data-storage-or-data-processing/#comment-201873 It is not quite correct to compare "normal" databases with google. Google is very special with special needs. For example a while ago I was searching google using some keywords to test my homepage ranking in search results. I was quite surprised for a phenomenon that simultaneous searches in my home computer and work computer (which physically are away ~4 km from each other) returned different results. Obviously they used different google sites and one of them had not updated it's index. So can you imagine a relational database that will return different results for SUM of your company salaries depending on which node you'll execute your query? :) I have Oracle background and it's RAC (real application clusters) option is a step towards your proposed solution, but it uses the same storage and it always gives back consistent results, not depending on which node you run your statement. It is both it's strength and weakness. Strength because it gives back on all nodes the same result, weakness because that means all nodes have to get most recent versions of database blocks and in case of frequent changes it puts a quite big load on interconnect and waits are introduced. It is not quite correct to compare “normal” databases with google. Google is very special with special needs. For example a while ago I was searching google using some keywords to test my homepage ranking in search results. I was quite surprised for a phenomenon that simultaneous searches in my home computer and work computer (which physically are away ~4 km from each other) returned different results. Obviously they used different google sites and one of them had not updated it’s index. So can you imagine a relational database that will return different results for SUM of your company salaries depending on which node you’ll execute your query? :)
I have Oracle background and it’s RAC (real application clusters) option is a step towards your proposed solution, but it uses the same storage and it always gives back consistent results, not depending on which node you run your statement. It is both it’s strength and weakness. Strength because it gives back on all nodes the same result, weakness because that means all nodes have to get most recent versions of database blocks and in case of frequent changes it puts a quite big load on interconnect and waits are introduced.

]]> By: bluecells » Blog Archive » MySQL: Data Storage or Data Processing http://www.mysqlperformanceblog.com/2007/11/18/mysql-data-storage-or-data-processing/comment-page-1/#comment-198715 bluecells » Blog Archive » MySQL: Data Storage or Data Processing Tue, 20 Nov 2007 00:35:08 +0000 http://www.mysqlperformanceblog.com/2007/11/18/mysql-data-storage-or-data-processing/#comment-198715 [...] can read the full story here Filed under: [...] [...] can read the full story here Filed under: [...]

]]> By: peter http://www.mysqlperformanceblog.com/2007/11/18/mysql-data-storage-or-data-processing/comment-page-1/#comment-197751 peter Mon, 19 Nov 2007 09:27:34 +0000 http://www.mysqlperformanceblog.com/2007/11/18/mysql-data-storage-or-data-processing/#comment-197751 AlexN, This is not the thing I'm talking about. The proper cursors allow you to stream date easily but they do not store the main problem. They are still not parallel. AlexN,

This is not the thing I’m talking about. The proper cursors allow you to stream date easily but they do not store the main problem. They are still not parallel.

]]> By: AlexN http://www.mysqlperformanceblog.com/2007/11/18/mysql-data-storage-or-data-processing/comment-page-1/#comment-197703 AlexN Mon, 19 Nov 2007 08:04:28 +0000 http://www.mysqlperformanceblog.com/2007/11/18/mysql-data-storage-or-data-processing/#comment-197703 There is, for example, Mondrian (http://mondrian.pentaho.org/), the OLAP tool that uses MySQL as a DB engine for ROLAP. About 50% of all processing job is done on server. It is not very fast, however. From my experience, the main advantage in data processing on server using MSSQL (not MySQL) is gained form its cursors, that use row level locks and do not interfere with data load and retrieval. Ideal for background processes. In MySQL cursor syntax is obscure, and I think that many people still are not using them at all. There is, for example, Mondrian (http://mondrian.pentaho.org/), the OLAP tool that uses MySQL
as a DB engine for ROLAP. About 50% of all processing job is done on server. It is not very fast,
however. From my experience, the main advantage in data processing on server using MSSQL (not MySQL)
is gained form its cursors, that use row level locks and do not interfere with data load and retrieval.
Ideal for background processes. In MySQL cursor syntax is obscure, and I think that many people
still are not using them at all.

]]> By: peter http://www.mysqlperformanceblog.com/2007/11/18/mysql-data-storage-or-data-processing/comment-page-1/#comment-197091 peter Sun, 18 Nov 2007 18:15:16 +0000 http://www.mysqlperformanceblog.com/2007/11/18/mysql-data-storage-or-data-processing/#comment-197091 Yeah. Good quote though I do not think so. I think the current technologies are not mature enough and may be they will develop to lead the next wave but it is also possible some other leaders pop up. Yeah. Good quote though I do not think so. I think the current technologies are not mature enough and may be they will develop to lead the next wave but it is also possible some other leaders pop up.

]]> By: Brian Aker http://www.mysqlperformanceblog.com/2007/11/18/mysql-data-storage-or-data-processing/comment-page-1/#comment-197067 Brian Aker Sun, 18 Nov 2007 17:53:53 +0000 http://www.mysqlperformanceblog.com/2007/11/18/mysql-data-storage-or-data-processing/#comment-197067 Hi! You are probably not surprised to know that some of us are advocating this. I'll probably blog what my MySQL internal thoughts sometime soon :) A quote I heard recently "Hadoop will replace the database vendors". Good quote aye? Cheers, -Brian Hi!

You are probably not surprised to know that some of us are advocating this. I’ll probably blog what my MySQL internal thoughts sometime soon :)

A quote I heard recently “Hadoop will replace the database vendors”. Good quote aye?

Cheers,
-Brian

]]>