Using the SeleniumBase module in Python, how to get URL of link that changes with each query?

[dan-ieljin]

Here is the HTML from a website I'm working with:

<pre>
Unique sequence file: 
<a href="/cgi-bin/elimdupesv2/elimdupes.cgi?id=4ROEFYUD&amp;output=unique_default.Fasta">View</a> 
<a href="/cgi-bin/elimdupesv2/elimdupes.cgi?id=4ROEFYUD&amp;download=unique_default.Fasta">Download</a>
<pre>

What's displayed:

Unique sequence file: View Download

The download link changes every run because a different file is produced. How should I extract the URL and save it into a variable to download the file? I tried this but it didn't work:

class Example(BaseCase):

def test_(self):
    ...
    url = self.find_link_text('Download', timeout=None).get_link_attribute()
    self.download_file(file_url, destination_folder=None)

Thanks a bunch in advance.

[mdmintz]

@bsmith4133 Try:

href = self.get_link_attribute("Download", "href")
if href.startswith('/'):
    url = self.driver.current_url
    domain_url = self.get_domain_url(url)
    href = domain_url + href
self.download_file(href)

(The default downloads folder is "./downloaded_files")

[dan-ieljin]

@mdmintz Thanks for the great solution. Do you have any recommendations for how to download from something like this, which doesn't contain a URL but instead a function of some sort? It doesn't open a link but instead downloads automatically.

<input type="image" src="files/download.png" onclick="startDownload();">

The interface looks like this:

Or would I have to just ask the user for the path to the downloaded file for the next steps? (This is part of a pipeline.)

[mdmintz]

Hi @bsmith4133 ,
You might be able to see where startDownload(); is defined by calling source = self.get_page_source() and then using regular Python string parsing to extract the URL from the method definition. If necessary, there are methods such as self.execute_script(JAVASCRIPT) that can help you with that. Based on that one line of html that you posted, this is the best advice that I can offer at this time.

[dan-ieljin]

@mdmintz Thanks so much for the suggestion. Here is what I found for the method definition from this website. There doesn't seem to be a URL here, but not sure if this makes anything more clear, as I'm not completely familiar with JavaScript:

function startDownload() {
    var e = jQuery.extend(!0, {}, ajaxObj);
    e.data[ridKey] = ctrlHandle,
    jTotalInputs.each(function(t) {
        !this.disabled && this.checked && (e.data[this.name] = this.value)
    }),
    ajaxStart(e, '<span style="text-decoration: blink;">Formatting data ...</span>')
}
var pinger_loaded = !1, reqObjects = {}, dataWaiting = 0, jCgiMsgHolder, jDLTable, jDLDiv, jStatusDiv, jTicker, jTimer, jRunTime, jTtlQs, jsTtlVQs, jsTtlFs, jsTtlNoHits, jTtlCDs, jTtlCLs, jTtlSpFeats, jTtlGnFeats, jTotalInputs, jTDataHits, jTDataAligns, jTDataFeats, jAlnFmtAsn, jAlnFmtXml, jAlnFmtJson, jAlnFmtText, jDModeRep, jDModeStd, jDModeFull, jCLOnly, jQDefl, jCDDefl, jXmpTblMsg, jXmpTable, jDLForm, jDLRid, navWin, timer = new tmrControl;
jDocument.ready(function() {
    jCgiMsgHolder = $("#msgwrap"),
    jTicker = $("#div_ticker"),
    jTimer = $("#p_timer"),
    timer.jqContainer = jTimer,
    jCdsId = $("#id_cdsid"),
    jRunTime = $("#id_rtime"),
    jTtlQs = $("#id_ttlQs"),
    jsTtlVQs = $("#id_ttlVQs"),
    jsTtlFs = $("#id_ttlFs"),
    jsTtlNoHits = $("#id_ttlNHs"),
    jTtlCDs = $("#id_ttlCDs"),
    jTtlCLs = $("#id_ttlCLs"),
    jTtlSpFeats = $("#id_ttlSpFeats"),
    jTtlGnFeats = $("#id_ttlGnFeats"),
    jStatusDiv = $("#div_Status"),
    jDLTable = $("#tbl_DLPanel"),
    jDLDiv = $("#div_DLPanel"),
    jTotalInputs = $("input", jDLTable),
    jTotalInputs.each(function(e) {
        var t = $(this);
        t.click(syncDLPanel);
        switch (t.attr("id")) {
        case "hid_tdata_hits":
            jTDataHits = t;
            break;
        case "hid_tdata_aligns":
            jTDataAligns = t;
            break;
        case "hid_tdata_feats":
            jTDataFeats = t;
            break;
        ...
        }
    }),
    jXmpTblMsg = $("#td_prompt"),
    jXmpTable = $("#td_XmpTable"),
    jDLForm = $("#frm_Dld"),
    jDLRid = $("#hid_DlRid"),
    syncDLPanel(),
    jDocument.scroll(hOnDocScroll),
    jWindow.resize(hOnDocScroll),
    hOnDocScroll();
    var e = jQuery.extend(!0, {}, ajaxObj);
    xmpHandle ? (e.data[ridKey] = ctrlHandle + "-" + xmpHandle,
    ajaxStart(e, '<span style="text-decoration: blink;">Getting results ...</span>')) : (e.data[ridKey] = ctrlHandle,
    e.data[tDataKey] = NOTARGET_LIT,
    ajaxStart(e, '<span style="text-decoration: blink;">Searching...</span>')),
    __sysInfoBox.hideDelay = 400,
    __sysInfoBox.smartSize = !1,
    $(__sysInfoBox.topContainer).css({
        width: 600,
        height: 400
    })
});

Do you have any recommendations for this? I'd be happy to provide more code or anything else if needed.

[mdmintz]

I'm not exactly sure how to extract the URL you're looking for out of that. There's probably an easier way to accomplish what you're going for. I'm currently working on several projects at once.