April 16, 2014

Flexviews is a working scalable database transactional memory example

http://Flexvie.ws fully implements a method for creating materialized views for MySQL data sets. The tool is for MySQL, but the methods are database agnostic. A materialized view is an analogue of software transactional memory. You can think of this as database transactional memory, or as database state distributed over time, but in an easy way to manage.

It has been shown that combinatorial algebraics can be applied to all aggregate functions (Flexviews uses only composable operations even for deletes over “non-distributable aggregate functions”), and Flexviews includes this capability today. Because Flexviews can move database objects forward in time synchronously with each other to specific points in time in database state, invalid database state is not possible, particularly if there are thorough database checks.

Flexviews is:
lock free for read
fully log based
supports all DML
supports all aggregate functions
supports join
supports NULL values
supports simple projection
only rolls forward in time right now
MVCC
row level locking with innodb

Flexviews was based on the following research:
pages.cs.wisc.edu/~beyer/
citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.41.7788

Also includes my own personal and extensive research into combinatorial algebra over non-distributable aggregate functions. I was not aware of research into combinatorial algebra and developed these techniques in isolation from other researchers as I am an amateur.

About Justin Swanhart

Comments

  1. business daily says:

    Less hassle identifying and promoting a new master for fail-over even better read load balancing – my last proposal for the future of the PHP replication plugin….What?…Think of a global transaction ID as a unique identifier for a change set in a database cluster. The PHP mysqlnd replication and load balancing plugin could learn to do the very same for you…..Create a MEMORY table which is replicated with a single UNSIGNED BIGINT and increment it at the end of each transaction…CREATE TABLE trx .trx id BIGINT UNSIGNED NOT NULL. ENGINE memory ..INSERT INTO trx VALUES 0 ..When ever you commit a transaction UPDATE the trx id field ..UPDATE trx SET trx id trx id 1.From ….Dear MySQL database administrator please drop us a note if you want this to be implemented in the PHP plugin! Partitioning itself has some CPU overheads but its almost negligible in the real workloads compared to disk i o overheads.In MySQL 5.0 or earlier versions where range partitioning is not supported creating daily weekly monthly tables is a good way as a workaround though applications have to aware of table name differences.CREATE TABLE message 201103 ..CREATE TABLE message 201104 ..CREATE TABLE message 201105 ..How about Database Sharding?As you know database sharding is very common approach for handling huge data.

  2. Justin Swanhart says:

    Sharing information makes Percona happy. It is the only thing that brings happiness. These are my gifts to the world so that all may understand that:

    The only paradox in the world is that the more you give, the more you get in terms of happiness. Sharing brings happiness, not having things.

    The people at Percona are some of the happiest people in the world. So much light that they brought us out of darkness.

  3. vegivamp says:

    Justin,

    I’ve seen quite a few of your articles in the last week or so. The subjects and tools you broach are quite interesting, but, no offence meant, I keep getting the impression that you’re about to send me a price list.

    Your writing style very much feels like advertising, at least to me, and I don’t find it very enjoyable to read. Might want to watch that. Apart from that, as I said, you broach interesting subjects, and I’ll definitely be having a look at your Flexviews tool.

Speak Your Mind

*