I like mysql/RDBMS because if the transaction handling, indexing capability, and “information density”, which is a 4 byte integer takes up 4 bytes, and is not converted to its string representation.

On the other hand, I like memcache, redis, tokyotyrant, and other such key/value stores, because of the performance and distribution (the document storage format)

I think some of the core of the problem you are talking about depends on how you look at the data in the database.

If you look at the database as an “interesting” form of data storage, then the objects (or tables…) are probably already in the format you are talking about.

If you look at the database as a sets of data (foreign keys, 3rd/boyce codd normal form, etc..), then your idea looks like it might make a good model front end, especially if you can solve the replication delay issue.

i’m wondering, why there’s no extended mysql-query-cache, which allows you to use cached results, even the table was alreasy updated. i think that would be much faster than to realize this via php and memcached.

I think Active cache is kind of merged API and Cache layer, so the point is your application should be only calling that code. I liked the Gearman concept here as it allows to glue multiple applications together – think for some objects constructed by Java code while others by Perl

Back to the original topic, I think that being able to specify what gets cached, and how it’s cached, is fairly important – there’s no way that mysql (or by extension any cache) can automatically understand what the application is trying to do. This is (sort of) the fundamental reason why things like memcached exist, and also the reason why the query caches in MySql dont work particularly well. (dont shoot me for that last comment, i’m sure they work just fine for some people)

If you could push some caching logic down closer to the data, then it might be useful in a number of circumstances, but at the end of the day you still need to be able to control what goes into your cache and what doesnt – which is really only something that you’d want the application to handle unless you’re supremely confident in your ability to write caching software at the database level.

The hard part is finding a balance between the two – retaining application control, and proximity to the data.
- Memcached sits on the application side of the fence.
- MySql has it’s own caches which sit completely on the other side.
- The ‘Active Cache’ would sit somewhere between the two – the question is where and how.

One could argue that memcached is “replacing” the data-tier of your architecture, but in reality it is only evolving it to handle certain cases differently (e.x., hot data -> memcached, cold data -> MySQL).

Going on to what Peter is saying here, I believe it is a middle-ware abstraction of a lot of redundant communications that could happen between the logic-tier and data-tier of your architecture.

If created properly, the logic would still happen in the logic-tier, and the data would still be handled in the data-tier, it is just that the data-tier would be aware of what type of data it is handling and therefore be able to store and return it efficiently.

Naturally, any idea taken to any extreme may result in problems, but I believe what Peter is suggesting could have some serious chances if the core idea is realized to its fullest.

Darius

You are describing what a number of people have been looking at creating

Not a crazy idea at all, the problem is picking the correct interface (aka… how will someone use this).

Cheers,
-Brian

History tend to repeat itself with slight variations. Modern shift to Web Based technologies is conceptually very similar to use of terminals to mainframe in 60s-70s. NoSQL existed in form of non relational databases, there are many such examples.

In general a lot of things done for years in J2EE are coming to LAMP in modified form, tuned for a lot more adhoc form.

Designing three tier application is one thing, evolving simply PHP (for example) application by adding memcache, gearman or any other components to take care of important issues in a simple way is completely different.

It’s probably not for everyone, but it was just a random idea of the sort of things that might be possible for a system that’s “loosely aware” of the underlying data structures being used.

I think in a general sense, i still prefer to go for the “other approach” of storing cooked data in memcached if possible, rather than raw query data. It doesnt really make sense in all circumstances though.
In one specific case, in our system there is a specific object which is frequently used and takes 5 queries to load. Despite we could cache the 5 queries individually, we’d rather just cache the object as it’s capable of invalidating itself from the cache, because it saves us a lot of overhead.

Have you considered that your active cache suggestion could potentially be implemented by somehow extending the mysql client infrastructure?
ie. mysql_query(“SELECT * FROM X”, ….., cache=true);
That way you could provide seamless integration for those people that want a simple but highly effective caching solution.