Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Needs max file size and max runtime flags/switches #12

Closed
gprime31 opened this issue Apr 4, 2023 · 6 comments · Fixed by #20
Closed

Needs max file size and max runtime flags/switches #12

gprime31 opened this issue Apr 4, 2023 · 6 comments · Fixed by #20
Assignees
@RamanaReddy0M
Labels
Status: Completed Nothing further to be done with this issue. Awaiting to be closed. Type: Enhancement Most issues will probably ask for additions or changes.
Milestone
alterx v0.0.2

Comments

@gprime31
Copy link

gprime31 commented Apr 4, 2023

Subs.txt contains 15k lines
Results in 100+GB file with the enrich flag
So a max file output size and a max runtime switch would be great. Other than that the tool is awesome!!
@tarunKoyalwar
Copy link
Member

@gprime31 , alterx already has limit flag to limit number of results

   -limit int      limit the number of results to return (default 0)

@gprime31
Copy link
Author

gprime31 commented Apr 4, 2023

@gprime31 , alterx already has limit flag to limit number of results

   -limit int      limit the number of results to return (default 0)

That's fine but how many results is say 52.4 GB? See the problem.

@tarunKoyalwar
Copy link
Member

tarunKoyalwar commented Apr 4, 2023 •
edited
Loading

@gprime31 , on an average size of single subdomain) is ~30 Bytes (or even less) . so for 52.4 GB it would be approx 1.746 Billion . not sure about the use case but I get your point and we will try to add option -max-size

@gprime31
Copy link
Author

gprime31 commented Apr 4, 2023

appreciate it.

@ehsandeep ehsandeep added the Type: Enhancement Most issues will probably ask for additions or changes. label Apr 6, 2023
@gprime31
Copy link
Author

gprime31 commented Apr 10, 2023 •
edited
Loading

Did some testing and here are some rough values for reference:

-	-limit 30000000 	= ~ 1.7GB
-	-limit 100000000 	= ~ 5.4GB
-	-limit 200000000	= ~ 11.5GB
-	-limit 250000000	= ~ 14.7GB
-	-limit 500000000	= ~ 29.3 GB
-	-limit 1000000000 	= ~ 52.2GB

if anyone else needs it.
subdomain list = 15K subs.

@RamanaReddy0M RamanaReddy0M self-assigned this May 17, 2023
RamanaReddy0M added a commit that referenced this issue May 17, 2023
@RamanaReddy0M
Add max file size support (#12)
1182377
@RamanaReddy0M RamanaReddy0M mentioned this issue May 17, 2023
Merged
@RamanaReddy0M RamanaReddy0M linked a pull request May 17, 2023 that will close this issue
Add max file size support #20
Merged
ehsandeep added a commit that referenced this issue May 30, 2023
@RamanaReddy0M @ehsandeep
Add max file size support (#20)
b6e46a7 
* Add max file size support (#12)

* rename dedube temp directory
fix removing generated temp dir on early exit

* update readme

* Add size units to max size flag

* update golfags
use sizevar
remove deprecated version callback

* dep update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
@ehsandeep ehsandeep added the Status: Completed Nothing further to be done with this issue. Awaiting to be closed. label May 30, 2023
@ehsandeep ehsandeep added this to the alterx v0.0.2 milestone May 30, 2023
@ehsandeep
Copy link
Member

@gprime31 this is now supported in latest release - https://github.com/projectdiscovery/alterx/releases/tag/v0.0.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@RamanaReddy0M RamanaReddy0M

Labels
Status: Completed Nothing further to be done with this issue. Awaiting to be closed. Type: Enhancement Most issues will probably ask for additions or changes.
Projects
None yet
Milestone
alterx v0.0.2
Development

Successfully merging a pull request may close this issue.

Add max file size support projectdiscovery/alterx
4 participants
@ehsandeep @gprime31 @tarunKoyalwar @RamanaReddy0M