多线程测试有重复序列号

[huxiaofei]

public class SnowFlakeKeyTest2 {
private static Set keys = new HashSet();
public static void main(String[] args) throws Exception {
for(int i=0;i<100;i++) {
SnowFlakeKeyTest2.GetKey runner = new SnowFlakeKeyTest2().new GetKey();
new Thread (runner).start();;
}
System.out.println(keys);

}

class GetKey implements Runnable{

	@Override
	public void run() {
		SnowFlake key = new SnowFlake(30,20);
		long k = key.nextId();
		if(!keys.contains(k)) {
			keys.add(k);
		}else {
			System.out.println("重复:"+k);
		}
	}
}

}

运行结果，有很多重复

重复:329755153370267648
重复:329755153370267648
重复:329755153370267648
重复:329755153370267648
重复:329755153374461952
重复:329755153374461952
重复:329755153374461952
重复:329755153374461952
重复:329755153378656256
重复:329755153378656256
重复:329755153378656256
重复:329755153378656256
重复:329755153382850560
重复:329755153391239168
重复:329755153391239168
重复:329755153391239168
重复:329755153391239168
重复:329755153395433472
重复:329755153395433472
重复:329755153395433472
重复:329755153395433472
重复:329755153395433472
重复:329755153399627776
重复:329755153399627776
重复:329755153399627776
重复:329755153399627776
重复:329755153399627776
重复:329755153399627776
重复:329755153399627776
重复:329755153399627776
重复:329755153399627776
重复:329755153399627776
重复:329755153395433472
重复:329755153395433472
重复:329755153395433472
重复:329755153395433472
重复:329755153395433472
重复:329755153403822080
重复:329755153412210688
重复:329755153412210688
重复:329755153412210688
重复:329755153420599296
重复:329755153420599296
重复:329755153424793600
重复:329755153424793600
重复:329755153424793600
重复:329755153424793600
重复:329755153424793600
重复:329755153424793600
重复:329755153424793600
重复:329755153424793600
重复:329755153424793600
重复:329755153424793600
重复:329755153424793600
重复:329755153424793600
重复:329755153433182208
[329755153424793600, 329755153433182208, 329755153403822080, 329755153370267648, 329755153374461952, 329755153378656256, 329755153382850560, 329755153391239168, 329755153395433472, 329755153399627776, 329755153408016384, 329755153412210688, 329755153420599296, 329755153428987904]
重复:329755153433182208
重复:329755153433182208
重复:329755153433182208
重复:329755153433182208
重复:329755153433182208
重复:329755153433182208
重复:329755153437376512
重复:329755153437376512
重复:329755153437376512
重复:329755153437376512
重复:329755153441570816
重复:329755153441570816
重复:329755153441570816
重复:329755153441570816
重复:329755153441570816
重复:329755153441570816
重复:329755153441570816
重复:329755153441570816
重复:329755153445765120
重复:329755153445765120

[hartmut-co-uk]

Hi @huxiaofei, you need to initialise each SnowFlake instance with a different tuple of <datacenterId, machineId>.

You can generate up to
12bit = 4096 unique IDs
per datacenterId
per machineId
per millisecond

Valid values for datacenterId & machineId are 5bit each -> 0...31

Try following instead:

package com.example.demo.utils;

import java.util.HashSet;
import java.util.Set;

class SnowFlakeKeyTest {
    private static Set keys = new HashSet();

    public static void main(String[] args) {
        long start = System.currentTimeMillis();

        for (int i = 0; i < 31; i++) {
            for (int j = 0; j < 31; j++) {
                SnowFlakeKeyTest.GetKey runner = new SnowFlakeKeyTest().new GetKey(i, j);
                new Thread(runner).start();
            }
        }

        long end = System.currentTimeMillis();
        System.out.println(keys.size() + " unique ids have been generated in " + (end - start) + "ms!\n" + keys);

    }

    class GetKey implements Runnable {

        private final int datacenterId;
        private final int machineId;

        public GetKey(int datacenterId, int machineId) {
            this.datacenterId = datacenterId;
            this.machineId = machineId;
        }

        @Override
        public void run() {
            SnowFlake key = new SnowFlake(datacenterId, machineId);
            long k = key.nextId();
            if (!keys.contains(k)) {
                keys.add(k);
            } else {
                System.out.println("重复:" + k);
            }
        }
    }
}

[huxiaofei]

thank s for @hartmut-co-uk reply.

在实际部署时，纵然有很多的数据中心，上百台服务器，具体到某一个服务器上，
datacenterID & machineId 就是一个常量。
此时在多线程并发请求产生ID，就会出现重复。

[hartmut-co-uk]

Hi @huxiaofei
datacenterId & machineId shouldn't be constant but different for each instance of SnowFlake. On the individual instance multi-threading is a common scenario. You need to instantiate SnowFlake as a singleton / bean. The implementation itself is thread safe.

If you can't guarantee singletons by tuple <datacenterId, machineId> you logically will have duplicates.

If you actually have big data / high volumes of IDs to generate (hundreds of servers?) - you'd need to provide IDs as-a-service and request IDs via API. So you can run a cluster of your 'ID Service' (with different configuration datacenterId & machineId) and request new IDs from your servers/applications via API.

[beyondfengyu]

上面说得很明白，一个节点就一个单例的SnokeFlake

[hartmut-co-uk]

Hi, still not a real Unit Test - but I've put together a more proper example of above's example (concurrency/synchronization) which I think might be worth sharing...

import java.util.*;
import java.util.concurrent.atomic.AtomicInteger;

class SnowFlakeKeyTest {
    private static Set keys = Collections.synchronizedSet(new HashSet(9610000));
    private static AtomicInteger x = new AtomicInteger(0);

    public static void main(String[] args) {
        long start = System.currentTimeMillis();

        List<Thread> threads = new ArrayList<>(961);
        for (int i = 0; i < 31; i++) {
            for (int j = 0; j < 31; j++) {
                SnowFlakeKeyTest.GetKey runner = new SnowFlakeKeyTest().new GetKey(i, j);
                Thread thread = new Thread(runner);
                thread.start();
                threads.add(thread);
            }
        }

        System.out.println(threads.size() + " threads have been started...");

        try {
            System.out.println("Waiting for threads to finish...");
            for (Thread thread: threads) {
                thread.join();
            }
            long end = System.currentTimeMillis();
            System.out.println(keys.size() + " unique ids (should: " + x + ") have been generated in " + (end - start) + "ms!");
        } catch (InterruptedException e) {
            System.out.println("Main thread Interrupted");
        }
    }

    class GetKey implements Runnable {

        private final int datacenterId;
        private final int machineId;

        public GetKey(int datacenterId, int machineId) {
            this.datacenterId = datacenterId;
            this.machineId = machineId;
        }

        @Override
        public void run() {
            SnowFlake key = new SnowFlake(datacenterId, machineId);

            for (int i = 0; i < 10000; i++) {
                long k = key.nextId();
                if (!keys.add(k)) {
                    System.out.println("duplicate:" + k);
                }
                x.incrementAndGet();
            }
        }
    }
}

Output:

961 threads have been started...
Waiting for threads to finish...
9610000 unique ids (should: 9610000) have been generated in 3757ms!