Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Examples and/or more education on how to use this ? #201

Open
mcc85s opened this issue Jul 19, 2019 · 9 comments
Open

Examples and/or more education on how to use this ? #201

mcc85s opened this issue Jul 19, 2019 · 9 comments

Comments

@mcc85s
Copy link

mcc85s commented Jul 19, 2019

I've been wanting to get into runspacing for some time, although, I'm not certain what can fit inside the script block or not.

As it is, I've got too much on my plate to continue testing with it, but one thing I'd like to do is get this idea working with some loops... similar to how the threads are controlled by an array declaration... like 0..5 or etc.

Any chance someone could be kind enough to show me some examples or use case scenarios on what can be done, or what can't be? I found an older video from a couple of years ago that talks about Runspace Factory, and I can only assume that PoshRSJobs is a very well refined derivative of that methodology.

Feel free to drop me a line if you'd like, mcook@securedigitsplus.com or check out my GitHub repo's... "personal account repo" @ github.com/mcc85s/PSD-Remaster or "company account repo" github.com/secure-digits-plus-llc/hybrid-desiredstatecontroller

Any feedback in terms of PoshRSJobs would be greatly appreciated, but if you have other suggestions, I'm always looking to learn more.

  • Michael C.
@copdips
Copy link
Contributor

copdips commented Jul 20, 2019

hi @mcc85s

Use case, suppose we want to get the computer's local administraors membership over 100 servers.

And we have the script to do it over one server. Then, instead of running the script one by one ( a loop on the 100 servers), we can use poshrsjob to do it at once. You will get the same result but much more faster.

@mcc85s
Copy link
Author

mcc85s commented Jul 25, 2019

I understood that part, and that's fine.

I'm looking to start more processes on a single machine, for instance, downloading multiple files simultaneously.

I could use some more education on multi-threading in powershell in general, but what I'm looking to do is to essentially spin up several file transfers at once concurrently instead of running them in serial.

Question is, can I do that with an array or hashtable ? And if so, what are some examples or limitations?

Most of the information I see out there is for SQL servers... not useful when I don't really use invoke-command or what have you.

@MVKozlov
Copy link
Contributor

Get-Command -Module BitsTransfer
Without RSJob :)

or $urllist | Start-RSJob { any-command-to-download $_ }

@mcc85s
Copy link
Author

mcc85s commented Jul 25, 2019

Interesting... Kind of lines up with the project I've been working on for MDT. I've only been working with PS for 7 months but i've been relentless about it...

I'm still learning quite a lot about it.

Can it be used to parse the registry as well ?

@copdips
Copy link
Contributor

copdips commented Jul 25, 2019

You can almost do anything with RSJob.

But parsing registry seems looking up into some files (registry), I think you can do it in a mono-thread without RSJob, it should be fast enough.

@mcc85s
Copy link
Author

mcc85s commented Jul 25, 2019

Your suggestions have reduced the wait time to like a hell of a lot less...
I want to like, give you a hug.
But if you're like 'nah bro... that's ok... I'm good on the hugs department'... Then here's a Fist bump.

@mcc85s
Copy link
Author

mcc85s commented Jul 25, 2019

The BitsTransfer module seems to work great for what I'm trying to do.

But I'm interested in how to feed this part here.
"$urllist | Start-RSJob { any-command-to-download $_}"

I'd really like to do more with it. I just don't fully understand it yet.

I'm trying to pass multiple variables into that job. That $_ sign there seems to mean that it will pull a verbatim value, which is not what I'm trying to do.

In my research, I see some limitations on being able to pass a hash table or other .net elements into a job or runspace. In fact, it's probably the one thing that I haven't truly figured out yet. Not to mention, there appears to be quite a difference in using $_ vs $anything else, but that's beside the point. ( i think in that aspect it's working as an object rather than by a count... )

But case in point, I'll give you an example of how I'm trying to use multiple runspaces

- - - - - - - - - - - -#

I have a script that pulls all of the elements of this hash table...

    $Item = @( 0..15 )
    0..15 | % { $Item[$_] = @{      
    Name = "$( $DisplayName[$_] ) v$( $Version[$_] )"
    File = "$( $Target[$_] )\$( $File[$_] ).$( $Exe[$_] )"
    URL = "$( $URL[$_] )"
    Checksum = "$( $Checksum[$_] )"} 

- - - - - - -

... and returns the value $Item if the function is called

Then the process actually validates the checksum for each file and if it's not valid it'll toss that file away and report that.

In that $Item, I have those URL's as $Item.URL
If I try to feed that example above, and I use IWR then I won't have an -OutFile which is important.

So for the above example...

How can I utilize

- - - - - - -

ForEach ( $i in $item )
{ $Item | Start-RSJob { command $_[$i] } }

- - - - - - -

Because the $i will keep the variables synchronized and each time I've tried to test with this module, I get a lot of errors and things don't work.

Now I'm all for experimentation, it's kind of my thing now... but I may need to see an example of how I could do this before I'm able to grasp it's core elements... I have a lot of things I'm concurrently working on and this has been a back burner thing for a while...

But I suppose the BitsTransfer module gets me a lot closer to what I'm looking to do.

@MVKozlov
Copy link
Contributor

your example

ForEach ( $i in $item )
{ $Item | Start-RSJob { command $_[$i] } }

gave me a headache :)
seems you want to write something like that

ForEach ( $i in $item )
{ $i | Start-RSJob { command $_ } }

It's really hard to understand what you want to achieve but if you try to use objects it may be better than hash
my own methods looks like this

#prepare objects
$objects = [PSCustomObject]@{
Name = 'aaa'
Value1 = 'bbb'
Value2 = 'ccc'
Sum = ''
},
[PSCustomObject]@{
Name = 'zzz'
Value1 = 'xxx'
Value2 = 'yyy'
$sum = ''
}
# start jobs
$jobs = $objects | Start-RSJob {
  param($o)
  # some work
  $o.SUm = $o.Value1 + $o.Value2
  #
  # result output
  $o
}
#wait for jobs completion and get results
$results = $jobs | Wait-RSJob | Receive-RSJob
$jobs | Remove-RSJobs
# use results
$results | Export-Csv

But if you need to interact with you data while your jobs still running you should use some sort of synchronized hash

here's th @proxb blog post about it
https://learn-powershell.net/2015/06/11/supporting-synchronized-collections-in-poshrsjob/

@mcc85s
Copy link
Author

mcc85s commented Jul 26, 2019

I checked out his article. I played around with it a little bit...

Didn't realize I had posted the thing in question before so I pulled my last comment. As far as using what you put down? I think that illustrates how to use it.
I'll let you know how I make out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants