Stored Function to generate Sequences

April 2008
M	T	W	T	F	S	S
« Mar		May »
	1	2	3	4	5	6
7	8	9	10	11	12	13
14	15	16	17	18	19	20
21	22	23	24	25	26	27
28	29	30

Posted by peter |

Today a customer asked me to help them to convert their sequence generation process to the stored procedure and even though I have already seen it somewhere I did not find it with two minutes of googling so I wrote a simple one myself and posting it here for public benefit or my later use

PLAIN TEXT

SQL:

delimiter //
CREATE FUNCTION seq(seq_name char (20)) returns int
begin
UPDATE seq SET val=last_insert_id(val+1) WHERE name=seq_name;
RETURN last_insert_id();
end
//
delimiter ;
CREATE TABLE `seq` (
`name` varchar(20) NOT NULL,
`val` int(10) UNSIGNED NOT NULL,
PRIMARY KEY (`name`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1
INSERT INTO seq VALUES('one',100);
INSERT INTO seq VALUES('two',1000);

This implementation uses single table to maintain multiple sequences which application can use in desired way, though you of course could have them using different tables.

I use MyISAM tables here which allows to use such sequences in transactions without serializing transactions which require access to same sequence, though it is not as safe as if you use Innodb table in this case.

Even though implementation is just 2 lines of code it seems to confuse a lot of people because last_insert_id() is used rather unusual way - with argument. This way of using this function allows you to "inject" the value to be returned next time this function is called.

Sometimes people wonder why you would like to use sequences instead of MySQL auto_increment columns ? Leaving aside more exotic ways of sequences even pure sequential value as in the case above can be quite helpful - in MySQL 5.0 you may with to use them instead of auto_increment with Innodb tables to avoid short term "table level locks" which innodb sets when Insert is happening in the table with auto_increment values. It is also helpful if you need to decouple ID generation from storing the data - for example IDs are generated on central server and when stored on number of servers or to number of individual tables.
Finally your own sequences allow you to generate multiple sequence values on demand with single statement for one or more sequences, which also can be helpful:

PLAIN TEXT

SQL:

mysql> SELECT seq('one'),seq('two'),seq('one'),seq('one');
+------------+------------+------------+------------+
| seq('one') | seq('two') | seq('one') | seq('one') |
+------------+------------+------------+------------+
| 102 | 1002 | 103 | 104 |
+------------+------------+------------+------------+
1 row IN SET (0.00 sec)

I should note this sequence generation requires serialization, though it is short term but it may still become the bottleneck for application with high sequence use rate. For such heavy duty apps I would use another approaches - in particular allocating "ranges" of sequences and caching them in applications, using UUID_SHORT() and other methods which do not require global lock for each time new sequence value needs to be retrieved.

1. kL

If you change sequence name from varchar to char you’ll get fixed-width table (slightly faster).

It’s probably important that sequences are in MyISAM table that doesn’t support transactions – otherwise different threads would be isolated (I in ACID) and could come up with the same sequence value!

Comment :: April 3, 2008 @ 2:49 am

2. Toby

This technique is perfectly safe with InnoDB tables.

Comment :: April 3, 2008 @ 7:19 am

3. peter

kL,

Yes you can use char here or even better ENUM or INT if you’re hunting for last 0.01% of performance.

Regarding Innodb Tables it still works because updates to the same rows will be serialized and so you can’t get same values in different threads.

Comment :: April 3, 2008 @ 8:17 am

4. Arjen Lentz

Here was mine http://arjen-lentz.livejournal.com/34627.html (been there for a few years
Also incorporates a trigger so the app can be oblivious it’s not an AUTO_INCREMENT column.

Comment :: April 3, 2008 @ 6:34 pm

5. Arjen Lentz

kL, even with a sequence generator stored in MyISAM you would not see the same number in different threads.

Comment :: April 3, 2008 @ 6:36 pm

6. peter

Arjen,

Indeed insert on dup key updates saves you from initialization though I’m wondering what way would be the faster.
Though I guess there would not be any significant difference here.

Comment :: April 3, 2008 @ 6:41 pm

7. Justin Huff

I typically see the update step written like:
UPDATE sequence SET nValue=last_insert_id(nValue+1) WHERE strName=?

I’d like to allow servers the option of asking for more than one id per call by moving the increment step outside the last_insert_id:
UPDATE sequence SET nValue=last_insert_id(nValue)+? WHERE strName=?

The value in the table shifts from being the ‘last id handed out’ to ‘the next id to hand out’. That way requesters don’t need to know how big the last block was.

Am I missing something here?
Thanks!

Comment :: July 30, 2008 @ 6:22 pm

8. saroj

thanks peter code worked

Comment :: June 1, 2009 @ 10:15 pm

9. Bruno Braga

This code does not handle concurrency/locking. Has anyone tested this against overloaded systems? I have the feeling it will not work properly.

MySQL Performance Blog

April 2, 2008

11 Comments »

Trackbacks/Pingbacks