Exception in thread "main" org.jsoup.HttpStatusException: HTTP error fetching URL. Status=429

[coranark]

Hi,

I love the program but when i tryed using it today i got the following error.

Exception in thread "main" org.jsoup.HttpStatusException: HTTP error fetching URL. Status=429, URL=https://www.royalroad.com/fiction/34592/core-chronicles/chapter/532426/101
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:679)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:628)
at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:260)
at net.ruippeixotog.scalascraper.browser.JsoupBrowser.executeRequest(JsoupBrowser.scala:65)
at net.ruippeixotog.scalascraper.browser.JsoupBrowser$$anonfun$3.apply(JsoupBrowser.scala:76)
at net.ruippeixotog.scalascraper.browser.JsoupBrowser$$anonfun$3.apply(JsoupBrowser.scala:76)
at scala.Function1$$anonfun$andThen$1.apply(Function1.scala:52)
at scala.Function1$$anonfun$andThen$1.apply(Function1.scala:52)
at net.ruippeixotog.scalascraper.browser.JsoupBrowser.get(JsoupBrowser.scala:33)
at com.aivean.royalroad.Main$$anonfun$4$$anonfun$apply$1.apply(Main.scala:69)
at com.aivean.royalroad.Main$$anonfun$4$$anonfun$apply$1.apply(Main.scala:69)
at scala.util.Try$.apply(Try.scala:192)
at com.aivean.royalroad.Main$.retry(Main.scala:60)
at com.aivean.royalroad.Main$$anonfun$4.apply(Main.scala:69)
at com.aivean.royalroad.Main$$anonfun$4.apply(Main.scala:66)
at scala.collection.parallel.AugmentedIterableIterator$class.map2combiner(RemainsIterator.scala:115)
at scala.collection.parallel.immutable.ParVector$ParVectorIterator.map2combiner(ParVector.scala:62)
at scala.collection.parallel.ParIterableLike$Map.leaf(ParIterableLike.scala:1054)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply$mcV$sp(Tasks.scala:49)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$class.tryLeaf(Tasks.scala:51)
at scala.collection.parallel.ParIterableLike$Map.tryLeaf(ParIterableLike.scala:1051)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.internal(Tasks.scala:159)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.internal(Tasks.scala:443)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.compute(Tasks.scala:149)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:443)
at scala.concurrent.forkjoin.RecursiveAction.exec(RecursiveAction.java:160)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinTask.doJoin(ForkJoinTask.java:341)
at scala.concurrent.forkjoin.ForkJoinTask.join(ForkJoinTask.java:673)
at scala.collection.parallel.ForkJoinTasks$WrappedTask$class.sync(Tasks.scala:378)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.sync(Tasks.scala:443)
at scala.collection.parallel.ForkJoinTasks$class.executeAndWaitResult(Tasks.scala:426)
at scala.collection.parallel.ForkJoinTaskSupport.executeAndWaitResult(TaskSupport.scala:56)
at scala.collection.parallel.ExecutionContextTasks$class.executeAndWaitResult(Tasks.scala:558)
at scala.collection.parallel.ExecutionContextTaskSupport.executeAndWaitResult(TaskSupport.scala:80)
at scala.collection.parallel.ParIterableLike$ResultMapping.leaf(ParIterableLike.scala:958)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply$mcV$sp(Tasks.scala:49)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$class.tryLeaf(Tasks.scala:51)
at scala.collection.parallel.ParIterableLike$ResultMapping.tryLeaf(ParIterableLike.scala:953)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.compute(Tasks.scala:152)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:443)
at scala.concurrent.forkjoin.RecursiveAction.exec(RecursiveAction.java:160)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

[Aivean]

Seems that you hit the request rate limit of Royal Road: https://httpstatuses.com/429 .

Can you please wait a bit and try again, maybe using --from-chapter parameter to load book partially, if you don't need it in full. If that doesn't help, I'll add new options that limit the rate of requests (that, unfortunately, will work downloading slower).

[Aivean]

Drat, doesn't work for me too. Well, seems that we're going to limit the rate of requests after all.

[Aivean]

@coranark , please check https://github.com/Aivean/royalroad-downloader/releases/tag/2.2.0