Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Adding UUID v4 type keys. #37

Closed
wants to merge 6 commits into from

2 participants

@kevinmontuori

Hi Evan --

I have a use case where UUIDs and not serial integers are
required for the ID field throughout my DB. To facilitate this,
I've made a couple of changes to boss_db.git:

  • passing Options to the boss_db_adapter_mock and then to
    boss_db_mock_controller;

  • setting the IdCounter (the second piece of state data) in the
    mock_controller server to the atom "uuid" when the Option property
    "db_keytype" is set to "uuid";

  • generating a v4 UUID for unspecified Id fields;

  • changing the way ExistingId is parsed (to bypass tokenization)
    when the IdCounter state is set to "uuid".

Additionally, I've changed ChicagoBoss.git to add the property
"db_keytype" the Options sent to to boss_db:start/1. I'll submit a
pull request for that change and reference this one.

I've done preliminary testing with cb_admin and the scheme seems to be
working correctly for both {db_keytype, uuid} and an unspecified
db_keytype (i.e., it's backwards compatible). What I have not done
is:

  • tested relationships between models;

  • changed any of the external adapters (I have a need to update the
    pgsql adapter which seems straightforward, I haven't examined the
    others yet);

  • updated the documentation appropriately.

Before I went any further, I wanted to float the idea by you to see if
it's something you'd consider adding to the mainline and to double
check that the methodology is sound. Any advice is welcome!

Note that I'm using Travis Vachon's erlang-uuid library (BSD licensed
and I believe redistributable in this way) to generate the UUIDs.

Thanks for your consideration and for a really useful framework.

@kevinmontuori kevinmontuori referenced this pull request in ChicagoBoss/ChicagoBoss
Closed

Adding UUID v4 type keys. #153

@evanmiller
Owner

Hi Kevin, thanks for the patch! I like the idea and how you are going about it, db_keytype works fine. I wonder though if people will want to configure this at a per-table level? What is your experience? Maybe people could override the setting in the model with

-module(my_model, [Id::uuid(), ...]).

or

-module(my_model, [Id::serial(), ...]).

kevinmontuori added some commits
@kevinmontuori kevinmontuori added allowance for Id type information in model;
refactored record type validation to always validate id regardless of value or type
16b2a8f
@kevinmontuori kevinmontuori mock controller refactored to consider ::uuid() Id type dba7383
@kevinmontuori kevinmontuori fixed uuid keytype regression 8032697
@kevinmontuori kevinmontuori removed db_keytype property,
now relying on the ::uuid() type in the model spec.
7310041
@kevinmontuori kevinmontuori tweaked mock controller to not use uuid as an IdCounter anymore;
fixed up the news_controller to limit id splits to two parts, making uuids ok.
885228e
@kevinmontuori

Having reviewed Evan's comment about specifying keytype as part of the module/attributes definition I think that's a far better approach. I'm closing this pull request and will submit a different one based upon that scheme.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Sep 13, 2012
  1. @kevinmontuori

    added BSD licensed UUID library; added db_keytype option handling to …

    kevinmontuori authored
    …the mock controller and mock adapter
Commits on Sep 14, 2012
  1. @kevinmontuori

    added allowance for Id type information in model;

    kevinmontuori authored
    refactored record type validation to always validate id regardless of value or type
  2. @kevinmontuori
  3. @kevinmontuori
  4. @kevinmontuori

    removed db_keytype property,

    kevinmontuori authored
    now relying on the ::uuid() type in the model spec.
  5. @kevinmontuori

    tweaked mock controller to not use uuid as an IdCounter anymore;

    kevinmontuori authored
    fixed up the news_controller to limit id splits to two parts, making uuids ok.
This page is out of date. Refresh to see the latest.
View
64 src/boss_db.erl
@@ -266,36 +266,40 @@ validate_record(Record) ->
validate_record_types(Record) ->
Errors = lists:foldl(fun
({Attr, Type}, Acc) ->
- Data = Record:Attr(),
- GreatSuccess = case {Data, Type} of
- {undefined, _} ->
- true;
- {Data, string} when is_list(Data) ->
- true;
- {Data, binary} when is_binary(Data) ->
- true;
- {{{D1, D2, D3}, {T1, T2, T3}}, datetime} when is_integer(D1), is_integer(D2), is_integer(D3),
- is_integer(T1), is_integer(T2), is_integer(T3) ->
- true;
- {Data, integer} when is_integer(Data) ->
- true;
- {Data, float} when is_float(Data) ->
- true;
- {Data, boolean} when is_boolean(Data) ->
- true;
- {{N1, N2, N3}, timestamp} when is_integer(N1), is_integer(N2), is_integer(N3) ->
- true;
- {Data, atom} when is_atom(Data) ->
- true;
- {_Data, Type} ->
- false
- end,
- if
- GreatSuccess ->
- Acc;
- true ->
- [lists:concat(["Invalid data type for ", Attr])|Acc]
- end
+ case Attr of
+ id -> Acc;
+ _ ->
+ Data = Record:Attr(),
+ GreatSuccess = case {Data, Type} of
+ {undefined, _} ->
+ true;
+ {Data, string} when is_list(Data) ->
+ true;
+ {Data, binary} when is_binary(Data) ->
+ true;
+ {{{D1, D2, D3}, {T1, T2, T3}}, datetime} when is_integer(D1), is_integer(D2), is_integer(D3),
+ is_integer(T1), is_integer(T2), is_integer(T3) ->
+ true;
+ {Data, integer} when is_integer(Data) ->
+ true;
+ {Data, float} when is_float(Data) ->
+ true;
+ {Data, boolean} when is_boolean(Data) ->
+ true;
+ {{N1, N2, N3}, timestamp} when is_integer(N1), is_integer(N2), is_integer(N3) ->
+ true;
+ {Data, atom} when is_atom(Data) ->
+ true;
+ {_Data, Type} ->
+ false
+ end,
+ if
+ GreatSuccess ->
+ Acc;
+ true ->
+ [lists:concat(["Invalid data type for ", Attr])|Acc]
+ end
+ end
end, [], Record:attribute_types()),
case Errors of
[] -> ok;
View
25 src/boss_db_mock_controller.erl
@@ -48,14 +48,21 @@ handle_call({save_record, Record}, _From, [{Dict, IdCounter}|OldState]) ->
Type = element(1, Record),
TypeString = atom_to_list(Type),
{Id, IdCounter1} = case Record:id() of
- id -> {lists:concat([Type, "-", IdCounter]), IdCounter + 1};
+ id -> case keytype(Record) of
+ uuid -> {lists:concat([Type, "-", uuid:to_string(uuid:v4())]), IdCounter};
+ _ -> {lists:concat([Type, "-", IdCounter]), IdCounter + 1}
+ end;
ExistingId ->
- [TypeString, IdNum] = string:tokens(ExistingId, "-"),
- Max = case list_to_integer(IdNum) of
- N when N > IdCounter -> N;
- _ -> IdCounter
- end,
- {lists:concat([Type, "-", IdNum]), Max + 1}
+ case keytype(Record) of
+ uuid -> {ExistingId, IdCounter};
+ _ ->
+ [TypeString, IdNum] = string:tokens(ExistingId, "-"),
+ Max = case list_to_integer(IdNum) of
+ N when N > IdCounter -> N;
+ _ -> IdCounter
+ end,
+ {lists:concat([Type, "-", IdNum]), Max + 1}
+ end
end,
NewAttributes = lists:map(fun
({id, _}) ->
@@ -88,6 +95,10 @@ code_change(_OldVsn, State, _Extra) ->
handle_info(_Info, State) ->
{noreply, State}.
+
+keytype(Record) ->
+ proplists:get_value(id, Record:attribute_types(), unspecified).
+
do_find(Dict, Type, Conditions, Max, Skip, SortBy, SortOrder) ->
Tail = lists:nthtail(Skip,
lists:sort(fun(RecordA, RecordB) ->
View
14 src/boss_news_controller.erl
@@ -54,7 +54,7 @@ handle_call({set_watch, WatchId, TopicString, CallBack, UserInfo, TTL}, From, St
(SingleTopic, {ok, StateAcc, WatchListAcc}) ->
case re:split(SingleTopic, "\\.", [{return, list}]) of
[Id, Attr] ->
- [Module, IdNum] = re:split(Id, "-", [{return, list}]),
+ [Module, IdNum] = re:split(Id, "-", [{return, list}, {parts, 2}]),
{NewState1, WatchInfo} = case IdNum of
"*" ->
SetAttrWatchers = case dict:find(Module, StateAcc#state.set_attr_watchers) of
@@ -75,7 +75,7 @@ handle_call({set_watch, WatchId, TopicString, CallBack, UserInfo, TTL}, From, St
end,
{ok, NewState1, [WatchInfo|WatchListAcc]};
_ ->
- case re:split(SingleTopic, "-", [{return, list}]) of
+ case re:split(SingleTopic, "-", [{return, list}, {parts, 2}]) of
[_Module, _IdNum] ->
IdWatchers = case dict:find(SingleTopic, State#state.id_watchers) of
{ok, Val} -> Val;
@@ -96,7 +96,7 @@ handle_call({set_watch, WatchId, TopicString, CallBack, UserInfo, TTL}, From, St
end;
(_, Error) ->
Error
- end, {ok, State, []}, re:split(TopicString, ", +", [{return, list}])),
+ end, {ok, State, []}, re:split(TopicString, ", +", [{return, list}, {parts, 2}])),
case RetVal of
ok -> {reply, RetVal, NewState#state{
watch_dict = dict:store(WatchId,
@@ -133,7 +133,7 @@ handle_call({extend_watch, WatchId}, _From, State0) ->
{reply, RetVal, NewState};
handle_call({created, Id, Attrs}, _From, State0) ->
State = prune_expired_entries(State0),
- [Module | _IdNum] = re:split(Id, "-", [{return, list}]),
+ [Module | _IdNum] = re:split(Id, "-", [{return, list}, {parts, 2}]),
PluralModel = inflector:pluralize(Module),
{RetVal, State1} = case dict:find(PluralModel, State#state.set_watchers) of
{ok, SetWatchers} ->
@@ -156,7 +156,7 @@ handle_call({created, Id, Attrs}, _From, State0) ->
{reply, RetVal, State1};
handle_call({deleted, Id, OldAttrs}, _From, State0) ->
State = prune_expired_entries(State0),
- [Module | _IdNum] = re:split(Id, "-", [{return, list}]),
+ [Module | _IdNum] = re:split(Id, "-", [{return, list}, {parts, 2}]),
PluralModel = inflector:pluralize(Module),
{RetVal, State1} = case dict:find(PluralModel, State#state.set_watchers) of
{ok, SetWatchers} ->
@@ -182,7 +182,7 @@ handle_call({deleted, Id, OldAttrs}, _From, State0) ->
{reply, RetVal, State1};
handle_call({updated, Id, OldAttrs, NewAttrs}, _From, State0) ->
State = prune_expired_entries(State0),
- [Module | _IdNum] = re:split(Id, "-", [{return, list}]),
+ [Module | _IdNum] = re:split(Id, "-", [{return, list}, {parts, 2}]),
IdWatchers = case dict:find(Id, State#state.id_attr_watchers) of
{ok, Val} -> Val;
_ -> []
@@ -242,7 +242,7 @@ future_time(TTL) ->
MegaSecs * 1000 * 1000 + Secs + TTL.
activate_record(Id, Attrs) ->
- [Module | _IdNum] = re:split(Id, "-", [{return, list}]),
+ [Module | _IdNum] = re:split(Id, "-", [{return, list}, {parts, 2}]),
Type = list_to_atom(Module),
DummyRecord = boss_record_lib:dummy_record(Type),
apply(Type, new, lists:map(fun
View
3  src/boss_record_compiler.erl
@@ -36,6 +36,9 @@ process_tokens([{']',_},{')',_},{dot,_}|_]=Tokens, TokenAcc, Acc) ->
process_tokens([{'-',N}=T1,{atom,N,module}=T2,{'(',_}=T3,{atom,_,_ModuleName}=T4,{',',_}=T5,
{'[',_}=T6,{var,_,'Id'}=T7|Rest], TokenAcc, []) ->
process_tokens(Rest, lists:reverse([T1, T2, T3, T4, T5, T6, T7], TokenAcc), []);
+process_tokens([{'-',_N}=T1,{atom,_,module}=T2,{'(',_}=T3,{atom,_,_ModuleName}=T4,{',',_}=T5,
+ {'[',_}=T6,{var,_,'Id'}=T7,{'::',_},{atom,_,VarType},{'(',_},{')',_}|Rest], TokenAcc, []) ->
+ process_tokens(Rest, lists:reverse([T1, T2, T3, T4, T5, T6, T7], TokenAcc), [{'Id', VarType}]);
process_tokens([{',',_}=T1,{var,_,VarName}=T2,{'::',_},{atom,_,VarType},{'(',_},{')',_}|Rest], TokenAcc, Acc) ->
process_tokens(Rest, lists:reverse([T1, T2], TokenAcc), [{VarName, VarType}|Acc]);
process_tokens([H|T], TokenAcc, Acc) ->
View
59 src/lib/uuid.erl
@@ -0,0 +1,59 @@
+% Copyright (c) 2008, Travis Vachon
+% All rights reserved.
+%
+% Redistribution and use in source and binary forms, with or without
+% modification, are permitted provided that the following conditions are
+% met:
+%
+% * Redistributions of source code must retain the above copyright
+% notice, this list of conditions and the following disclaimer.
+%
+% * Redistributions in binary form must reproduce the above copyright
+% notice, this list of conditions and the following disclaimer in the
+% documentation and/or other materials provided with the distribution.
+%
+% * Neither the name of the author nor the names of its contributors
+% may be used to endorse or promote products derived from this
+% software without specific prior written permission.
+%
+% THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+% "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+% LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+% A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+% OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+% SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
+% TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+% PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+% LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+% NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+% SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+%
+-module(uuid).
+-export([v4/0, to_string/1, get_parts/1, to_binary/1]).
+-import(random).
+
+% Generates a random binary UUID.
+v4() ->
+ v4(random:uniform(round(math:pow(2, 48))) - 1, random:uniform(round(math:pow(2, 12))) - 1, random:uniform(round(math:pow(2, 32))) - 1, random:uniform(round(math:pow(2, 30))) - 1).
+v4(R1, R2, R3, R4) ->
+ <<R1:48, 4:4, R2:12, 2:2, R3:32, R4: 30>>.
+
+% Returns a string representation of a binary UUID.
+to_string(U) ->
+ lists:flatten(io_lib:format("~8.16.0b-~4.16.0b-~4.16.0b-~2.16.0b~2.16.0b-~12.16.0b", get_parts(U))).
+
+% Returns the 32, 16, 16, 8, 8, 48 parts of a binary UUID.
+get_parts(<<TL:32, TM:16, THV:16, CSR:8, CSL:8, N:48>>) ->
+ [TL, TM, THV, CSR, CSL, N].
+
+% Converts a UUID string in the format of 550e8400-e29b-41d4-a716-446655440000
+% (with or without the dashes) to binary.
+to_binary(U)->
+ convert(lists:filter(fun(Elem) -> Elem /= $- end, U), []).
+
+% Converts a list of pairs of hex characters (00-ff) to bytes.
+convert([], Acc)->
+ list_to_binary(lists:reverse(Acc));
+convert([X, Y | Tail], Acc)->
+ {ok, [Byte], _} = io_lib:fread("~16u", [X, Y]),
+ convert(Tail, [Byte | Acc]).
Something went wrong with that request. Please try again.