E2E test: torrent upload

[josecelano]

E2E test for torrent upload.

Strategy

I want to:

• Generate a random torrent file on-the-fly
• Select the torrent and submit the form
• Assert that the app redirects to the torrent detail page with the right metadata: title. description, category, ...

Current status

It works with a fixed torrent file fixture.

Example of the test passing on CI: https://github.com/torrust/torrust-index-frontend/actions/runs/5546141538/jobs/10126011522?pr=185#step:7:684

But the problem is I want to generate a random torrent file like in the backend to be able to run the test many times. If you try to rerun the same test, it will fail because the torrent already exists in the database.

In the backend, we solved it by using a console command to generate the torrent file. Here it's not possible because with Cypress, we cannot use:

• Asynchronous functions in custom commands
• Use packages which need to access the filesystem.

Since it's run in the browser,

Screenshot

[josecelano]

It's only the first iteration. I'm using a fixed torrent file in the fixtures folder. I want to dynamically generate a new torrent file for each test execution so that I can isolate the test execution from previous executions. That was how we did it in the backend. With the current solution, you cannot execute the test twice because you cannot upload the torrent with the same infohash or title twice.

We could remove the torrent after executing the tests, but the test could fail for many reasons that do not depend on the subject under the test. For example, the backend API could be temporarily not available. For those cases, if the torrent was already uploaded but not removed, I would like to be able to re-run the test.

In the backend, we can run tests with a completely isolated environment but that's impossible here because we are using docker. In the backend, we just launch another child process accessing a different database.

I will try to find a node package to build the torrent in memory and try to write the new torrent in the fixtures folder as described in the Cypress documentation:

If that's not possible, we could try to use an API. You can also get files from APIs:

https://docs.cypress.io/api/commands/selectfile#From-an-API-response

Maybe we could create a new service with docker for the E2E test env with an API to generate torrent files. We could use the create-torrent package or the console command we are using in the backed: intermodal.

But I will try first the previous solution because it's much simpler.

[josecelano]

I'm trying to generate the torrent file on-the-fly with a custom command like this:

import createTorrent from 'create-torrent';

Cypress.Commands.add("generateFixture", () => {
  // First, I write the new file
  cy.writeFile('cypress/fixtures/torrents/sample.txt', "hello world!");

  // Then I read the file
  cy.readFile('cypress/fixtures/torrents/sample.txt', 'utf8').then((content) => {

    const buf = Buffer.from(content);
    buf.name = 'sample.txt.torrent';

    createTorrent(buf, (err, torrent) => {

      let buffer = Cypress.Buffer.from(torrent);

      if (!err) {
        cy.writeFile('cypress/fixtures/torrents/sample.txt.torrent', buffer, 'binary');
    })  
  })
});

But I cannot use callbacks because of a Cypress restriction.

I've been discussing it with ChatGPT:

https://chat.openai.com/share/99d2ebdc-613c-4a84-8281-bfc2fe82bc14

And there are some alternatives:

Option 1

Build our API service to generate torrent files on-the-fly.

We can create a docker image for the service and add it to the docker composer configuration for the E2E testing environment.

Option 2

Add that feature to the application. I would work only for tiny files. But It needs to make sense for the end-user.

It would be something like this: https://kimbatt.github.io/torrent-creator/

Pros

• We do not need an extra service

Cons:

• In general, it's not useful for most users.

Option 3

Use an independent env for each test, like in the backend. So there is no need to generate random data. We could always use the pre-built fixtures in the repo.

• Cons

I think it would be too slow with docker.

Option 4

Delete the torrent after running the test.

Cons:

• I have the impression that we could end up with a lot of inconsistencies. And test would also be slower because we have to clean the database after running them.

Pros:

• The database would always be clean, so no need to clean it from time to time in local dev env.

Conclusion

The option suggested by ChatGPT seems to be easy to implement, and we do not need to add features to the app only for testing. Maybe it could be useful for other cases.

Links

• How to create fixtures with random data using Cypress and faker

What do you think @da2ce7?

[Wolfremium13]

Hello, I would like to ask/give an opinion. From what I see, I'm not sure if it would generate much noise to generate that file in the before of the tests directly instead of using custom commands. On the other hand, I'm not sure if I've misunderstood, but is the idea to generate a Buffer or use .selectFile once we have created it? Because in my opinion, re-reading the file seems to be more the responsibility of the browser.

I'm not exactly sure how complex it can be to generate a torrent, but it seems like a better idea not to use createTorrent, which has a dependency on a plain text file, but instead use a Builder torrentMetadataBuilder (this is an example name) that uses parameters so that we can understand how the content for the <random-name|specified-name>.torrent file is generated following the option 2. Additionally, with the use of the builder, we can retrieve some parameters that we can later use in the expected output, such as relevant metadata.

Example using the upload test.

  it("should be able to upload a torrent", () => {
    const fileName = "test.torrent"
    const filePath = `${randomFilePath}/{fileName}`;
    const randomContent = torrentMetadataBuilder().addTitle("test").build(); // Invented code
    cy.writeFile(filePath, randomContent);

    cy.visit("/upload");
    cy.get("input[data-cy=\"upload-form-title\"]").type("title-mandelbrot_set_01.torrent");
    cy.get("textarea[data-cy=\"upload-form-description\"]").type("description");
    cy.get("select[data-cy=\"upload-form-category\"]").select("software");
    cy.get("input[data-cy=\"upload-form-torrent-upload\"]").selectFile(
      {
        contents: filePath,
        fileName: fileName,
        mimeType: "application/x-bittorrent"
      }, { force: true });
    cy.get("button[data-cy=\"upload-form-submit\"]").click();
    
    cy.url().should("include", "/torrent/*"); // I should don't care about the uuid that BE creates for me

Finally, in the after section, the files in cypress/fixtures/torrents/random/ should be cleaned up. I have provided another path in case you don't want to delete the reference file mandelbrot_set_01.torrent and the database (random users included), so as not to disturb the preconditions of other tests. I hope this helps a bit.

[josecelano]

Hi @Wolfremium13 thank you for your feedback!

Hello, I would like to ask/give an opinion. From what I see, I'm not sure if it would generate much noise to generate that file in the before of the tests directly instead of using custom commands.

The reason I want to use a custom command is because I want to reuse the command in all tests that require upload a torrent. I want to have something like this:

const torrent_file_name = random(); // For example: torrent_392429878.torrent
const torrent_data_file_contents = "hello";
cy.generateRandomTorrentFromContents(`cypress/fixtures/torrents/torrent_file_name`, torrent_data_file_contents);

If I tested it manually, the process would be:

• Create a new txt file in a temp dir: random_file392429878.txt
• Write a random text or the ID into the file so we can even write asserts in the future if we test torrent clients using the same file. The file would contain the text 392429878.
• Create the torrent file for that content. From random_file392429878.txt we get a new file random_file392429878.torrent, which is the torrent file for the original data file.
• Go to the upload page, fill in the form and upload the file random_file392429878.torrent

That's how I did it for the API. See here.

    #[tokio::test]
    async fn it_should_allow_authenticated_users_to_upload_new_torrents() {
        let mut env = TestEnv::new();
        env.start(api::Version::V1).await;

        // ...

        let uploader = new_logged_in_user(&env).await;
        let client = Client::authenticated(&env.server_socket_addr().unwrap(), &uploader.token);

        let test_torrent = random_torrent();
        let info_hash = test_torrent.info_hash().clone();

        let form: UploadTorrentMultipartForm = test_torrent.index_info.into();

        let response = client.upload_torrent(form.into()).await;

        let uploaded_torrent_response: UploadedTorrentResponse = serde_json::from_str(&response.body).unwrap();

        assert_eq!(
            uploaded_torrent_response.data.info_hash.to_lowercase(),
            info_hash.to_lowercase()
        );
        assert!(response.is_json_and_ok());
    }

I do not want to generate the test torrent in the before function, I want to do it in the test itself but abstracting away the whole process of creating a new random torrent file (dynamic fixture).

On the other hand, I'm not sure if I've misunderstood, but is the idea to generate a Buffer or use .selectFile once we have created it? Because in my opinion, re-reading the file seems to be more the responsibility of the browser.

I want to use .selectFile. The reason I was generating a buffer was I cannot write directly to the filesystem using createTorrent, but you can pass a buffer. It's an indirection because with Cypress you cannot access the filesystem. I planned to generate the buffer and then use cy.writeFileto write the buffer content into the fixture file. It does not work anyway because createTorrent uses callbacks which do not work in Cyppress. See Commands Are Asynchronous in Cypress docs

I'm not exactly sure how complex it can be to generate a torrent, but it seems like a better idea not to use createTorrent, which has a dependency on a plain text file, but instead use a Builder torrentMetadataBuilder (this is an example name) that uses parameters so that we can understand how the content for the <random-name|specified-name>.torrent file is generated following the option 2. Additionally, with the use of the builder, we can retrieve some parameters that we can later use in the expected output, such as relevant metadata.

That would be my preferred solution, but I have not found a package that I can use with Cypress, which requires:

• No async
• To be executable on the browser

But since I only want to create very simple torrents, maybe I should explore this solution.

I'm also considering another solution. In the backend, I'm using a console command to generate torrents: intermodal. CYpress has an exec command to run system command. Maybe I can do the same as I did in the backend. I had a similar problem; I could not find a library to create torrent files on the fly.

Example using the upload test.

  it("should be able to upload a torrent", () => {
    const fileName = "test.torrent"
    const filePath = `${randomFilePath}/{fileName}`;
    const randomContent = torrentMetadataBuilder().addTitle("test").build(); // Invented code
    cy.writeFile(filePath, randomContent);

    cy.visit("/upload");
    cy.get("input[data-cy=\"upload-form-title\"]").type("title-mandelbrot_set_01.torrent");
    cy.get("textarea[data-cy=\"upload-form-description\"]").type("description");
    cy.get("select[data-cy=\"upload-form-category\"]").select("software");
    cy.get("input[data-cy=\"upload-form-torrent-upload\"]").selectFile(
      {
        contents: filePath,
        fileName: fileName,
        mimeType: "application/x-bittorrent"
      }, { force: true });
    cy.get("button[data-cy=\"upload-form-submit\"]").click();
    
    cy.url().should("include", "/torrent/*"); // I should don't care about the uuid that BE creates for me

That example is precisely what I'm trying to achieve but I want to reuse this code in other tests:

const fileName = "test.torrent"
const filePath = `${randomFilePath}/{fileName}`;
const randomContent = torrentMetadataBuilder().addTitle("test").build(); // Invented code
cy.writeFile(filePath, randomContent);

Regarding using the torrent infohash in the assertion:

cy.url().should("include", "/torrent/*"); 

I wanted to check that the torrent is not only uploaded but also parsed correctly.

Finally, in the after section, the files in cypress/fixtures/torrents/random/ should be cleaned up. I have provided another path in case you don't want to delete the reference file mandelbrot_set_01.torrent and the database (random users included), so as not to disturb the preconditions of other tests. I hope this helps a bit.

Yes, I should delete the newly generated files, but do you think I should also delete the torrent in the database? I prefer to reset the E2E test env from time to time because I have the impression that tests would be much slower if you have a lot of them.

And finally, another thing to consider is that we do not need to create a temporary file. It seems you can selectFile directly from a buffer.

cy.get('input[type=file]').selectFile({
  contents: Cypress.Buffer.from('file contents'),
  fileName: 'file.txt',
  mimeType: 'text/plain',
  lastModified: Date.now(),
})

The final solution could be something like:

  it("should be able to upload a torrent", () => {
    const contentsFileName = "`text-${randomId}.txt`;
    const contentsFilePath = `cypress/torrents/{contentsFileName}`;
    const randomContent = `${randomId`;
    cy.writeFile(contentsFilePath, randomContent);
    cy.exec(`imdl torrent create --input ${contentsFilePath} --ouput ${contentsFilePath}.torrent`)

    cy.visit("/upload");
    cy.get("input[data-cy=\"upload-form-title\"]").type("title-mandelbrot_set_01.torrent");
    cy.get("textarea[data-cy=\"upload-form-description\"]").type("description");
    cy.get("select[data-cy=\"upload-form-category\"]").select("software");
    cy.get("input[data-cy=\"upload-form-torrent-upload\"]").selectFile(
      {
        contents: `${contentsFilePath}.torrent`,
        fileName: `${contentsFileName}.torrent`,
        mimeType: "application/x-bittorrent"
      }, { force: true });
    cy.get("button[data-cy=\"upload-form-submit\"]").click();
    
    cy.url().should("include", "/torrent/*");

But moving the fixture creation to a custom command. But I would prefer your proposal, a torrent builder. I will try that solution first.

[Wolfremium13]

Thanks for the feedback I appreciated a lot <3. I'll share a few opinions:

Yes, I should delete the newly generated files, but do you think I should also delete the torrent in the database?

In my opinion yes, to keep it the use case enviroment as cleaned as possible (if we could do it). If for example we're getting radom numbers for the name, the probability to get exactly the same number it's low, but could happen and generate an unexpected behaviour on a test that shouldn't fail. Could happen with users created on the same instance of the database in different places.

I wanted to check that the torrent is not only uploaded but also parsed correctly.

Seems like your asstertion it's tied to the behaviour of the implementation of back-end in order to get the number. Maybe I'm wrong but, it could generate problems if for example we change the name or number of the file because it's random.

[josecelano]

Hey @Wolfremium13

Thanks for the feedback I appreciated a lot <3. I'll share a few opinions:

Yes, I should delete the newly generated files, but do you think I should also delete the torrent in the database?

In my opinion yes, to keep it the use case enviroment as cleaned as possible (if we could do it). If for example we're getting radom numbers for the name, the probability to get exactly the same number it's low, but could happen and generate an unexpected behaviour on a test that shouldn't fail. Could happen with users created on the same instance of the database in different places.

OK. I'm not worried about the env on the CI because it's a new DB every time you run the tests, and there should not be collisions, however, leaving the DB clean I think it's a good thing and maybe not as slow as I expect. I can do it and remove it later of try to find another solution if tests are really slow.

I wanted to check that the torrent is not only uploaded but also parsed correctly.

Seems like your asstertion it's tied to the behaviour of the implementation of back-end in order to get the number. Maybe I'm wrong but, it could generate problems if for example we change the name or number of the file because it's random.

The number is a domain ID, the torrent infohash which is a hash that does not change, but we are considering adding other protocols in the future (not only BitTorrent) and in that case, we could switch to use our ID, so I totally agree, maybe I only have to check in the details page that the title, description, etcetera are OK, even the infohash but not in the URL unless we consider that URL a permalink and we want to test that it's available or something like that. I mean, we should test it not because we want to be sure the torrent was uploaded correctly as I said, but for other reasons. For now, it's an implementation detail as you said.

[da2ce7]

I think that creating a dynamic endpoint in the frontend something like /dev/test-torrent/get, that generates and provides a random torrent file would be the most elegant.

Since we have the version namespace we can put it in /dev/ instead of /v1/ so it shouldn't pollute things too much.

Edit: This isn't just like a fixture. Dealing with torrents is a core-property of this application.

[josecelano]

I think that creating a dynamic endpoint in the frontend something like /dev/test-torrent/get, that generates and provides a random torrent file would be the most elegant.

Since we have the version namespace we can put it in /dev/ instead of /v1/ so it shouldn't pollute things too much.

Edit: This isn't just like a fixture. Dealing with torrents is a core-property of this application.

Hi @da2ce7 I also think it's the most convenient. I think it's going to be used a lot for testing and maybe in the future it could be reused for a production feature, so I think it is worth the effort to build this new endpoint. And it should not be too difficult if we use a third-party library.

[josecelano]

Hi @Wolfremium13 @da2ce7, I've been thinking again about this problem and exploring the solution with a new dev endpoint and I have some new concerns:

• Although this new endpoint is not going to be used in production I think we should include it under the version namespace somehow. Test are coupled to a given API version and we could need to change the endpoint in the future. So I would use something like api/v1/dev/test-torrent
• I do not know if I should implement an endpoint to generate a random torrent like api/v1/dev/test-torrent/random or if we could pre-generate an ID for the new resource like api/v1/dev/test-torrent/UUID. Maybe it does not matter.
• In the current API we are using a console command imdl (intermodal) to generate the random torrents from random text files. If we follow the same approach we need to add that external dependency (console command) also for production. I do not like that option because I think we are changing too much the production code/requirements only for testing.
• Due to the previous point I was thinking if we should create a more generic endpoint where you can POST the data needed in the database to generate the torrent file. After all, we are uploading a torrent file, then we converted it into data that we store in the database, and finally we do the reverse process. This option could be more generic or useful but it makes harder to generate the torrent in the tests arrange phase. We need to calculate the pieces, etcetera, and that's not a trivial process. That was the reason I decided to use the imdl console command.
• For all those reasons, I decided that maybe the best solution would be to implement an independent solution: A testing API to generate sample torrents. My idea was to create a new repo with a simple web server app (using Axum) where you can call the api/v1/dev/test-torrent/random endpoint. We can generate a docker image for the webserver and It can be used to generate fixtures/random data for testing BitTorrent applications. This way we do not have to change the API production code and it could be reused in other repos that require its kind of sample data. This solution could take some time and I'm not sure if it's a good idea because maybe each repo has its own needs or maybe we need to change it very often while we are writing new E2E tests.

Conclusion

• I think the main problem is that the natural interface to generate torrent files is the file system. You usually create a torrent file from a file or directory. The original file can be big and hard to handle. And if you try to handle the data after processing the original content, that's even worse because you have the signatures.

• I'm trying to reconsider the whole thing because I suppose there must be a better solution. The reason I was using new fixtures on the fly is I did not want to have collisions if I wanted to run the same tests in the same environment. For example, if I run a test for uploading a torrent, I want to run it again without getting an error because the torrent already exists. But maybe that is not a good practice. Maybe as @Wolfremium13 mentioned, we should clean the database after running each test so you can run the test with the same data again. The only problem with this approach is we have to use a different fixture for each test because they can be executed in parallel. For example, we could have two tests for uploading a torrent with different asserts. At the most, we are going to have one torrent file per test, which is not too much. Another good side effect is when a test fails, you can easily check what was wrong in the database because the infohash would be the same (not the torrent ID).

What do you think? I would like to consider all the alternatives before implementing the dev endpoint. I think it could be easy to implement but not sure about maintainability.

[josecelano]

We can merge this after merging torrust/torrust-index#237

The workflow fails because the new endpoint version to generate random test torrents has not been merged yet into develop in the backend.

[josecelano]

We can merge this after merging torrust/torrust-index-backend#237

The workflow fails because the new endpoint version to generate random test torrents has not been merged yet into develop in the backend.

The workflow is passing because we've already merged the PR on the backend. The test works fine, and it's easy to read. We will probably extract a command to reuse the torrent generation and upload processes in other tests. But we can do it when we start working on adding other tests.

I will continue working on this PR. I'm deleting the test torrent when the test ends, but I also want to delete the torrent in the database as @Wolfremium13 suggested here.

[josecelano]

In order to delete the temp test torrent when the test ends, I need to get the infohash so that I can run SQL query, but I do not want to get the infohash from the UI because I do not want to couple this cleaning part of the test to the UI. THe only way to get the infohash is from the original uploaded torrent file, but we have a bug on the backend. The infohash of the indexed torrent does not match the infohash of the originally uploaded torrent. I need to fix that issue before continuing.

[josecelano]

In order to delete the temp test torrent when the test ends, I need to get the infohash so that I can run SQL query, but I do not want to get the infohash from the UI because I do not want to couple this cleaning part of the test to the UI. The only way to get the infohash is from the original uploaded torrent file, but we have a bug on the backend. The infohash of the indexed torrent does not match the infohash of the originally uploaded torrent. I need to fix that issue before continuing.

Hey @da2ce7, In the end, I'm using a different approach. Instead of parsing the downloaded torrent file, I've added a custom HTTP header to the download response. This solution is faster because we do not need to parse the file.

It has a side effect that I do not know if it is good. Since we are not parsing the file, we get the right infohash from the header even though the downloaded torrent has a different infohash. But in this test, we are not testing that the uploaded and downloaded torrents have the same infohash, we are only testing that you can upload a torrent, and after uploading it, you are redirected to the torrent details page.

By the way, I have been unable to parse the torrent file with a Cypress task. Whenever I try to add a task that calls an async function, I get an error saying that the CYpress config file is invalid. You have to return a promise, but I can't make it work.

But I really do not know why it was not working with the parse-torrent library. In fact, I'm using now an async function and it seems to work:

The Cypress config file:

import { defineConfig } from "cypress";
import { grantAdminRole } from "./cypress/e2e/contexts/user/tasks";
import { deleteTorrent } from "./cypress/e2e/contexts/torrent/tasks";
import { DatabaseConfig } from "./cypress/e2e/common/database";

function databaseConfig (config: Cypress.PluginConfigOptions): DatabaseConfig {
  return {
    filepath: config.env.db_file_path
  };
}

export default defineConfig({
  e2e: {
    baseUrl: "http://localhost:3000",
    setupNodeEvents (on, config) {
      on("task", {
        grantAdminRole: ({ username }) => {
          return grantAdminRole(username, databaseConfig(config));
        },
        deleteTorrent: ({ infohash }) => {
          return deleteTorrent(infohash, databaseConfig(config));
        }
      });
    }
  },
  env: {
    db_file_path: "./storage/database/torrust_index_backend_e2e_testing.db"
  }
});

And the function of the task:

// Custom tasks for user context

import { DatabaseConfig, DatabaseQuery, runDatabaseQuery } from "../../common/database";

// Task to grant admin role to a user by username
export const deleteTorrent = async (infohash: string, db_config: DatabaseConfig): Promise<boolean> => {
  try {
    await runDatabaseQuery(deleteTorrentQuery(infohash), db_config);
    return true;
  } catch (err) {
    return await Promise.reject(err);
  }
};

// Database query specifications

function deleteTorrentQuery (infohash: string): DatabaseQuery {
  return {
    query: "DELETE FROM torrust_torrents WHERE info_hash = ?",
    params: [infohash]
  };
}

As you can see, deleteTorrent is async, but I was not able to call the parse-torrent function inside a similar task for parsing the downloaded torrent.

The good thing is that while I was struggling to find a solution I took a look at the code inside the parse-torrent package, and the lines to extract the infohash from the torrent file are very simple:

https://github.com/webtorrent/parse-torrent/blob/master/index.js#L118-L145

  if (ArrayBuffer.isView(torrent)) {
    torrent = bencode.decode(torrent)
  }

  // sanity check
  ensure(torrent.info, 'info')
  ensure(torrent.info['name.utf-8'] || torrent.info.name, 'info.name')
  ensure(torrent.info['piece length'], 'info[\'piece length\']')
  ensure(torrent.info.pieces, 'info.pieces')

  if (torrent.info.files) {
    torrent.info.files.forEach(file => {
      ensure(typeof file.length === 'number', 'info.files[0].length')
      ensure(file['path.utf-8'] || file.path, 'info.files[0].path')
    })
  } else {
    ensure(typeof torrent.info.length === 'number', 'info.length')
  }

  const result = {
    info: torrent.info,
    infoBuffer: bencode.encode(torrent.info),
    name: arr2text(torrent.info['name.utf-8'] || torrent.info.name),
    announce: []
  }

  result.infoHashBuffer = await hash(result.infoBuffer)
  result.infoHash = arr2hex(result.infoHashBuffer)

I also tried to copy/paste that code instead of using the package, but I think there is a problem in this line:

result.infoHashBuffer = await hash(result.infoBuffer)

I do not know how to create a Cypress task which calls to a function defined elsewhere which contains an await. It seems using async/await with Cypress is tricky. I guess we will need to do that in the future for other tests, but for now, I prefer the alternative solution (the custom header with the infohash) because it's faster.

This PR can be merged after emerging torrust/torrust-index#243 now even if the torrust/torrust-index#242 issue is not solved yet.