Skip to content
This repository was archived by the owner on Mar 22, 2023. It is now read-only.

Mpsc queue docs#1119

Merged
igchor merged 3 commits intopmem:masterfrom
karczex:mpsc_queue_docs
Jul 9, 2021
Merged

Mpsc queue docs#1119
igchor merged 3 commits intopmem:masterfrom
karczex:mpsc_queue_docs

Conversation

@karczex
Copy link
Copy Markdown

@karczex karczex commented Jul 2, 2021

This change is Reviewable

Copy link
Copy Markdown
Contributor

@igchor igchor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 4 files reviewed, 2 unresolved discussions (waiting on @karczex)


examples/mpsc_queue/mpsc_queue.cpp, line 49 at r1 (raw file):

		[&](pmem::obj::experimental::mpsc_queue::batch_type rd_acc) {
			for (pmem::obj::string_view str : rd_acc) {
				std::cout << str.data() << std::endl;

Problem with printing garbage is that there is no null terminator. This is once again problem with no operator<< for string_view.


include/libpmemobj++/experimental/mpsc_queue.hpp, line 37 at r1 (raw file):

 * In case of crash or shutdown, reading and writing may be continued
 * by new process, without loss of any, already produced data.
 *

Please add that, try_consume_batch MUST be called after restart (otherwise produce might fail even if the queue is empty).

@codecov
Copy link
Copy Markdown

codecov Bot commented Jul 2, 2021

Codecov Report

Merging #1119 (4f2acb8) into master (8e5eec1) will decrease coverage by 0.01%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1119      +/-   ##
==========================================
- Coverage   94.24%   94.22%   -0.02%     
==========================================
  Files          52       52              
  Lines        5159     5159              
==========================================
- Hits         4862     4861       -1     
- Misses        297      298       +1     
Flag Coverage Δ
tests_clang_debug_cpp17 93.80% <ø> (-0.03%) ⬇️
tests_gcc_debug 90.81% <ø> (-0.22%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
include/libpmemobj++/experimental/mpsc_queue.hpp 95.37% <ø> (ø)
...ude/libpmemobj++/container/concurrent_hash_map.hpp 94.04% <0.00%> (-0.32%) ⬇️
include/libpmemobj++/transaction.hpp 82.35% <0.00%> (+0.84%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8e5eec1...4f2acb8. Read the comment docs.

@karczex karczex force-pushed the mpsc_queue_docs branch from 6a7e044 to bbfb334 Compare July 2, 2021 15:55
Copy link
Copy Markdown
Author

@karczex karczex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 4 files reviewed, 1 unresolved discussion (waiting on @igchor)


include/libpmemobj++/experimental/mpsc_queue.hpp, line 37 at r1 (raw file):

Previously, igchor (Igor Chorążewicz) wrote…

Please add that, try_consume_batch MUST be called after restart (otherwise produce might fail even if the queue is empty).

Done.

Copy link
Copy Markdown
Contributor

@igchor igchor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 2 files at r2.
Reviewable status: 1 of 4 files reviewed, 3 unresolved discussions (waiting on @karczex)


examples/mpsc_queue/CMakeLists.txt, line 5 at r2 (raw file):

cmake_minimum_required(VERSION 3.3)
project(inline_string CXX)

inline_string?


include/libpmemobj++/experimental/mpsc_queue.hpp, line 322 at r2 (raw file):

/**
 * Evaluates function f for the data, which is ready to be consumed.

Please also mention that try_consume_batch starts transaction internally (user callback is called within a transaction).


include/libpmemobj++/experimental/mpsc_queue.hpp, line 330 at r2 (raw file):

 *
 * see also: mpsc_queue::worker::try_produce()
 *

@throws transaction_scope_error ...

Copy link
Copy Markdown
Contributor

@KFilipek KFilipek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 4 files at r1, 1 of 2 files at r2.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @karczex)

@lukaszstolarczuk lukaszstolarczuk added this to the 1.13 milestone Jul 5, 2021
Copy link
Copy Markdown
Member

@lukaszstolarczuk lukaszstolarczuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 4 files at r1, 2 of 2 files at r2.
Reviewable status: all files reviewed, 30 unresolved discussions (waiting on @karczex)


examples/CMakeLists.txt, line 49 at r2 (raw file):

# Add developer checks
add_cppstyle(examples ${CMAKE_CURRENT_SOURCE_DIR}/*/*.*pp)
add_check_whitespace(examples ${CMAKE_CURRENT_SOURCE_DIR}/*.*)

you've missed on level - /*


examples/mpsc_queue/mpsc_queue.cpp, line 39 at r2 (raw file):

	pmem::obj::persistent_ptr<root> proot = pop.root();

	/* Create mpsc_queue runtime, which uses pmem_log_type object to store

at runtime (?)

I'm not sure what you had in mind


examples/mpsc_queue/mpsc_queue.cpp, line 43 at r2 (raw file):

	auto queue = pmem::obj::experimental::mpsc_queue(*proot->log, 1);

	/* Consume data, which was left in the queue from the previous run of

left ... from the prev. -> stored ... in the prev.


examples/mpsc_queue/mpsc_queue.cpp, line 44 at r2 (raw file):

	/* Consume data, which was left in the queue from the previous run of
	 * application */

the application


examples/mpsc_queue/mpsc_queue.cpp, line 78 at r2 (raw file):

			});
	}
	//! [try_produce_string_view]

did you check that nested snippets are working flawlessly?


examples/mpsc_queue/mpsc_queue.cpp, line 79 at r2 (raw file):

	}
	//! [try_produce_string_view]
	/* Porduce data to be consumed in next run of application. */

Porduce misspell


examples/mpsc_queue/mpsc_queue.cpp, line 79 at r2 (raw file):

	}
	//! [try_produce_string_view]
	/* Porduce data to be consumed in next run of application. */

the application


include/libpmemobj++/experimental/mpsc_queue.hpp, line 3 at r2 (raw file):

// SPDX-License-Identifier: BSD-3-Clause
/* Copyright 2021, Intel Corporation */

pls add \file description


include/libpmemobj++/experimental/mpsc_queue.hpp, line 35 at r2 (raw file):

 * queue.
 *
 * In case of crash or shutdown, reading and writing may be continued

continued from the last position...? pls extend


include/libpmemobj++/experimental/mpsc_queue.hpp, line 38 at r2 (raw file):

 * by new process, without loss of any, already produced data.
 *
 * note: try_consume_batch() MUST be called after restart (otherwise produce

what for must it be called? pls extend


include/libpmemobj++/experimental/mpsc_queue.hpp, line 38 at r2 (raw file):

 * by new process, without loss of any, already produced data.
 *
 * note: try_consume_batch() MUST be called after restart (otherwise produce

after restart of what?


include/libpmemobj++/experimental/mpsc_queue.hpp, line 101 at r2 (raw file):

public:
	/**
	 * Helper type, which allows to iterate over mpsc_queue's ranges via

helper type doesn't sound well, imho ;)

perhaps write it similar to below: Type representing...


include/libpmemobj++/experimental/mpsc_queue.hpp, line 118 at r2 (raw file):

	/**
	 * mpsc_queue producer worker class.
	 * note:  All workers should be destroyed before destruction of

should -> have to


include/libpmemobj++/experimental/mpsc_queue.hpp, line 121 at r2 (raw file):

	 * mpsc_queue
	 *
	 * see also: mpsc_queue:try_produce_batch()

Doxygen syntax: \see MyOtherClass::alloc()


include/libpmemobj++/experimental/mpsc_queue.hpp, line 154 at r2 (raw file):

	 * mpsc_queue.
	 *
	 * Object of this type have to be managed by pmem::obj::pool, to be

has to be


include/libpmemobj++/experimental/mpsc_queue.hpp, line 178 at r2 (raw file):

 *
 * @param[in] pmem reference to already allocated  pmem_log_type object
 * @param[in] max_workers maximum number of workers witch may be used by the

witch -> which


include/libpmemobj++/experimental/mpsc_queue.hpp, line 178 at r2 (raw file):

 *
 * @param[in] pmem reference to already allocated  pmem_log_type object
 * @param[in] max_workers maximum number of workers witch may be used by the

by the runtime? not in the runtime?


include/libpmemobj++/experimental/mpsc_queue.hpp, line 178 at r2 (raw file):

 *
 * @param[in] pmem reference to already allocated  pmem_log_type object
 * @param[in] max_workers maximum number of workers witch may be used by the

what do you mean may be used? if I try to create more than that, then what?


include/libpmemobj++/experimental/mpsc_queue.hpp, line 272 at r2 (raw file):

 * Constructs pmem_log_type object
 *
 * @param size size of the log

in what units?


include/libpmemobj++/experimental/mpsc_queue.hpp, line 280 at r2 (raw file):

/**
 * Returns string_view of the log, which allows to read-only access to the

what do you mean by the string_view of the log? pls clarify


include/libpmemobj++/experimental/mpsc_queue.hpp, line 308 at r2 (raw file):

/**
 * Registers the producer worker. Number of workers have to bo lesser or equal

less than or equal to


include/libpmemobj++/experimental/mpsc_queue.hpp, line 309 at r2 (raw file):

/**
 * Registers the producer worker. Number of workers have to bo lesser or equal
 * than specified in the mpsc_queue constructor.

specified what? pls extend e.g. ... max_workers specified in...


include/libpmemobj++/experimental/mpsc_queue.hpp, line 460 at r2 (raw file):

 * fail. Any exception thrown from f will result in terminate().
 *
 * @return true if f was evaluated, all data copied by it saved in the

were saved


include/libpmemobj++/experimental/mpsc_queue.hpp, line 461 at r2 (raw file):

 *
 * @return true if f was evaluated, all data copied by it saved in the
 * mpsc_queue, and visible for the consumer.

and are visible


include/libpmemobj++/experimental/mpsc_queue.hpp, line 463 at r2 (raw file):

 * mpsc_queue, and visible for the consumer.
 *
 * see also: try_consume_batch()

\see


include/libpmemobj++/experimental/mpsc_queue.hpp, line 510 at r2 (raw file):

 * the data is visible for the consumer. By default do nothing.
 *
 * @return true if f was evaluated, all data copied by it saved in the

.


include/libpmemobj++/experimental/mpsc_queue.hpp, line 639 at r2 (raw file):

/**
 * Returns an iterator to the beginning of the  accessed range of the

accessed range? what do you mean?

@karczex karczex force-pushed the mpsc_queue_docs branch from bbfb334 to 8f03709 Compare July 8, 2021 14:05
@karczex karczex force-pushed the mpsc_queue_docs branch from 8f03709 to 63a259c Compare July 8, 2021 14:21
Copy link
Copy Markdown
Author

@karczex karczex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 4 files reviewed, 30 unresolved discussions (waiting on @igchor, @karczex, @KFilipek, @lukaszstolarczuk, and @throws)


examples/CMakeLists.txt, line 49 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

you've missed on level - /*

Done.


examples/mpsc_queue/CMakeLists.txt, line 5 at r2 (raw file):

Previously, igchor (Igor Chorążewicz) wrote…

inline_string?

Done.


examples/mpsc_queue/mpsc_queue.cpp, line 39 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

at runtime (?)

I'm not sure what you had in mind

Done.


examples/mpsc_queue/mpsc_queue.cpp, line 43 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

left ... from the prev. -> stored ... in the prev.

Done.


examples/mpsc_queue/mpsc_queue.cpp, line 44 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

the application

Done.


examples/mpsc_queue/mpsc_queue.cpp, line 78 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

did you check that nested snippets are working flawlessly?

yes


examples/mpsc_queue/mpsc_queue.cpp, line 79 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

Porduce misspell

Done.


examples/mpsc_queue/mpsc_queue.cpp, line 79 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

the application

Done.


include/libpmemobj++/experimental/mpsc_queue.hpp, line 3 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

pls add \file description

Done.


include/libpmemobj++/experimental/mpsc_queue.hpp, line 35 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

continued from the last position...? pls extend

Done.


include/libpmemobj++/experimental/mpsc_queue.hpp, line 38 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

what for must it be called? pls extend

Done.


include/libpmemobj++/experimental/mpsc_queue.hpp, line 38 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

after restart of what?

Done.


include/libpmemobj++/experimental/mpsc_queue.hpp, line 101 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

helper type doesn't sound well, imho ;)

perhaps write it similar to below: Type representing...

Done.


include/libpmemobj++/experimental/mpsc_queue.hpp, line 118 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

should -> have to

Done.


include/libpmemobj++/experimental/mpsc_queue.hpp, line 121 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

Doxygen syntax: \see MyOtherClass::alloc()

Done.


include/libpmemobj++/experimental/mpsc_queue.hpp, line 154 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

has to be

Done.


include/libpmemobj++/experimental/mpsc_queue.hpp, line 178 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

witch -> which

Done.


include/libpmemobj++/experimental/mpsc_queue.hpp, line 178 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

by the runtime? not in the runtime?

Done.


include/libpmemobj++/experimental/mpsc_queue.hpp, line 280 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

what do you mean by the string_view of the log? pls clarify

Done.


include/libpmemobj++/experimental/mpsc_queue.hpp, line 308 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

less than or equal to

Done.


include/libpmemobj++/experimental/mpsc_queue.hpp, line 309 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

specified what? pls extend e.g. ... max_workers specified in...

Done.


include/libpmemobj++/experimental/mpsc_queue.hpp, line 322 at r2 (raw file):

Previously, igchor (Igor Chorążewicz) wrote…

Please also mention that try_consume_batch starts transaction internally (user callback is called within a transaction).

Done.


include/libpmemobj++/experimental/mpsc_queue.hpp, line 330 at r2 (raw file):

Previously, igchor (Igor Chorążewicz) wrote…

@throws transaction_scope_error ...

Done.


include/libpmemobj++/experimental/mpsc_queue.hpp, line 460 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

were saved

Done.


include/libpmemobj++/experimental/mpsc_queue.hpp, line 461 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

and are visible

Done.


include/libpmemobj++/experimental/mpsc_queue.hpp, line 463 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

\see

Done.


include/libpmemobj++/experimental/mpsc_queue.hpp, line 510 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

.

Done.


include/libpmemobj++/experimental/mpsc_queue.hpp, line 639 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

accessed range? what do you mean?

I have no better idea how to describe it.
I changed batch_type description little bit, so I believe it's understandable now.

@karczex karczex force-pushed the mpsc_queue_docs branch from 63a259c to 4f2acb8 Compare July 8, 2021 16:00
Copy link
Copy Markdown
Author

@karczex karczex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 4 files reviewed, 30 unresolved discussions (waiting on @igchor, @KFilipek, @lukaszstolarczuk, and @throws)


include/libpmemobj++/experimental/mpsc_queue.hpp, line 178 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

what do you mean may be used? if I try to create more than that, then what?

That's tricky ;) as currently there is an assert.


include/libpmemobj++/experimental/mpsc_queue.hpp, line 272 at r2 (raw file):

Previously, lukaszstolarczuk (Łukasz Stolarczuk) wrote…

in what units?

Done.

Copy link
Copy Markdown
Contributor

@igchor igchor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 1 of 4 files at r3.
Reviewable status: 1 of 4 files reviewed, 27 unresolved discussions (waiting on @igchor, @KFilipek, and @lukaszstolarczuk)

@igchor igchor merged commit 3c5f479 into pmem:master Jul 9, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants