Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logical stream iterators #28

Closed
mbrobbel opened this issue Mar 12, 2020 · 1 comment · Fixed by #45
Closed

Logical stream iterators #28

mbrobbel opened this issue Mar 12, 2020 · 1 comment · Fixed by #45
Assignees
Labels
🦀 implementation Item related to the implementation

Comments

@mbrobbel
Copy link
Contributor

mbrobbel commented Mar 12, 2020

Here are some suggestions to improve the API for logical streams:

/// Rename from LogicalStreamType
pub enum LogicalType {
  Null,
  Bits(Positive),
  Group(Group),
  Union(Union),
  Stream(Stream),
}

impl LogicalType {
  /// Returns an iterator over resulting split items. 
  pub fn split(&self) -> Split { ... }
  /// Returns an iterator over the physical streams resulting from splitting 
  /// and mapping the element streams to physical streams.
  pub fn physical(&self) -> PhysicalSplit { ... }
}

/// An iterator over split items from a LogicalType.
pub struct Split(indexmap::set::IntoIter<SplitItem>);

impl Iterator for Split {
  type Item = SplitItem;
}
impl DoubleEndedIterator for Split { ... }
impl ExactSizeIterator for Split { ... }

/// An iterator over physical split items from a LogicalType.
pub struct PhysicalSplit(indexmap::set::IntoIter<SplitItem>);

impl Iterator for PhysicalSplit {
  type Item = PhysicalSplitItem;
}
impl DoubleEndedIterator for PhysicalSplit { ... }
impl ExactSizeIterator for PhysicalSplit { ... }

/// An element stream with a path name and LogicalType. Contains no nested 
/// streams.
pub struct ElementStream {
  path_name: PathName,
  logical_type: LogicalType,
}

impl ElementStream {
  /// Returns the LogicalType of this element. Contains no nested streams.
  pub fn logical_type(&self) -> &LogicalType {
    &self.logical_type
  }
  /// Return all fields in this element stream
  pub fn fields(&self) -> Fields { ... }
}

impl From<ElementStream> for PhysicalStream { ... }

pub struct Signals(LogicalType);
impl Signals {
  /// Returns the LogicalType of this element.
  pub fn logical_type(&self) -> &LogicalType {
    &self.0
  }
  /// Returns all fields in these async signals.
  pub fn fields(&self) -> Fields { ... }
}

/// A split item is either an async signal (outside streamspace) or an element
/// stream (no nested streams).
pub enum SplitItem {
  Signals(Signals),
  Stream(ElementStream),
}

/// A split item is either an async signal (outside streamspace) or a physical
/// stream.
pub enum PhysicalSplitItem {
  Signals(Signal),
  Stream(PhysicalStream),
}

For the uses case of @ahadnagy this would result in something like this:

let haystack: LogicalType = ...;
let needle: &[Name] = ...;

let search: Option<SplitItem> = haystack
            .split()
            .find(|split_item|
                match split_item {
                    SplitItem::Signals(signals) => signals.fields(),
                    SplitItem::Stream(element_stream) => element_stream.fields(),
                }
                .keys()
                .windows(needle.len())
                .any(|name| name == filter)
            );

Given that we modify the Fields::keys method to return &[Name] instead of &PathName.

This also solves #27. We could also use a marker trait to distinguish between nested and unnested LogicalTypes.

@mbrobbel mbrobbel added the 🦀 implementation Item related to the implementation label Mar 12, 2020
@johanpel johanpel reopened this Mar 23, 2020
@johanpel
Copy link
Member

To-do leftovers:

- [ ] Documentation
- [ ] Tests
- [ ] Discuss API design

Follow-up PRs can incorporate these interfaces in the generator module.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🦀 implementation Item related to the implementation
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants