Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[C4GT] File Validator for cQube #1

Open
invinayak007 opened this issue May 11, 2023 · 26 comments
Open

[C4GT] File Validator for cQube #1

invinayak007 opened this issue May 11, 2023 · 26 comments

Comments

@invinayak007
Copy link
Collaborator

invinayak007 commented May 11, 2023 •
edited

Project Details:

This project aims to develop a File Validation utility with a user-friendly interface for cQube. The utility will perform checks files to be uploaded to ensure that the data meets the required specifications and is accurate and complete. Basically the Event file has to be validated before uploading that to cQube.

The key features to be implemented as part of this project are:

  • User-friendly interface: The utility should have an intuitive and user-friendly interface that allows the end user to easily upload and validate their files.
  • Validation rules: The File Validator utility should include validation rules that cover common file-related issues.
    • Format : Format of the data matches as per the spec defined earlier which is available in cQube
    • Completeness : All the required fields are filled
    • Accuracy
      • Dimension values : Dimension values are validated against the master data available in cQube
      • Metric values : Metric values should be within a defined range
  • Custom validation rules: The utility should allow the end user to define custom validation rules specific to their data. This would enable them to check for errors or inconsistencies that are unique to their data.
    • Ranges for certain fields
    • Pattern of a particular field
  • Error reporting: The File Validator utility should provide clear and detailed error messages when a file fails validation. This would help the end user identify and fix any issues with their data.

Learning Path:

Complexity: Moderate
Risk/Exploratory: Low
Skills Required: Python, familiarity with data validation techniques.
Name of Mentor: Vinayak Inamadar
Project size (in hours): 150-200 hours

Product Set Up:

To set up the project on a system, follow these guidelines:

  • Install the required programming language and libraries, such as Python and their respective data validation libraries.
  • Fork the project
  • Clone the forked project repository from GitHub.
  • Set up the environment variables and configurations required to run the utility.

For more details, go through https://github.com/Code4GovTech/C4GT

Acceptance Criteria:

The acceptance criteria for this project are:

  • The utility should be able to validate files uploaded by the end user and identify any errors or inconsistencies.
  • The utility should provide clear and detailed error messages when a file fails validation.
  • The utility should have an intuitive and user-friendly interface that allows the end user to easily upload and validate their files.

Out of Scope Elements:

The following elements are out of scope for this project:

  • Data cleaning and normalization.
  • Handling of large files that require distributed processing.
  • Integration with other systems or tools.

Key milestones for this project are:

  • Development of the File Validator utility with basic validation rules
  • Implementation of custom validation rules and error reporting
  • Coming up with UI for easy usage
  • Testing and debugging of the utility
@MridulMohanta19
Copy link

MridulMohanta19 commented May 12, 2023 •
edited

I am interested in contributing in this project as I am fairly proficient in the language and I want to work on this project.

@prakharrathi25
Copy link

I am also interested in this project. I am quite familiar with Python and it's related libraries and have contributed to open source projects using this tech stack.

@Srushhhti
Copy link

I am also interested in this project as I am well familiar with python and its libraries. I have worked upon some simple projects online and I would like to contribute here.

@lohith1392
Copy link
Contributor

I am interested in this project. I am quite familiar with Python and its usage in the backend like in Django, flask, and Machine learning. I want to work on this project and can include other features if time permits like duplicate detection and referential integrity.

@mahi01agarwal
Copy link

I am very interested in contributing to it. After reviewing the project description and codebase, I believe my skills and experience align well with the goals of the project. I would love the opportunity to work with you and your team to help bring this project to the next level.

@randomcuboid06
Copy link

I believe I will be able to contribute to this project! Looking forward to it😄

@Raghavi7124
Copy link

I'm interested in contributing to this project! As I believe that my skills would get aligned with this project!!!

@invinayak007
Copy link
Collaborator Author

Hi All, Good to know the response so far. Please start working on the proposals. If you have any queries please post it here. Thanks

@Srushhhti
Copy link

Hey, can you please provide me with the codebase for this project.

@harshinirhsv
Copy link

hi ,I'm interested in contributing to this project

@harshinirhsv
Copy link

we can start writing the proposal directly or need to resolve any issues?

@Komal-99
Copy link

I am also interested in this project as I have worked upon various projects related to python and data processing in ML field I would like to contribute here .

@invinayak007
Copy link
Collaborator Author

Thanks for your interest. Below is some detail on proposal writing.

What proposal denotes?
The basic motive behind the proposal is to analyse how proficient the student's understanding is regarding the project, along with their past work experiences and understanding about an open source.

How to start with proposal writing?
You can check this template for the proposal writing, can include these pointers in your proposal:
https://github.com/Code4GovTech/C4GT/wiki/Sample-Proposal

Before writing proposal, one should be clear with the implementation and the requriements of the projects, so try to interact with mentors check the previous discussions on the project channel of your interest, you can also interact with other contributors and can raise the joint PRs.

Note: After writing your proposal before submission, you can share it with your respective mentors of the project, they can share their feedback over it and you can incoporate that in your proposal.

All the best, Keep learning!

@Komal-99
Copy link

Thanks for your interest. Below is some detail on proposal writing.

What proposal denotes? The basic motive behind the proposal is to analyse how proficient the student's understanding is regarding the project, along with their past work experiences and understanding about an open source.

How to start with proposal writing? You can check this template for the proposal writing, can include these pointers in your proposal: https://github.com/Code4GovTech/C4GT/wiki/Sample-Proposal

Before writing proposal, one should be clear with the implementation and the requriements of the projects, so try to interact with mentors check the previous discussions on the project channel of your interest, you can also interact with other contributors and can raise the joint PRs.

Note: After writing your proposal before submission, you can share it with your respective mentors of the project, they can share their feedback over it and you can incoporate that in your proposal.

All the best, Keep learning!

Hi , Any communication channel for the project ?

@invinayak007
Copy link
Collaborator Author

Please go through https://unstop.com/competitions/code-for-govtech-2023-samagra-governance-675801

@masterismail
Copy link

masterismail commented May 23, 2023 •
edited

Hi @invinayak007 I've written a proposal for this project would you let me know the best channel to connect with you to get a review for the same.

@Ashutosh27102002
Copy link

Highly Interested, have done a similar project for submission on Yukti Portal (Government initiative to promote start-ups) and its currently in for Round 3

@Arjitm26
Copy link

Interested to contribute for the project!

@yash-0211
Copy link

Looking forward to contributing to this project :)

@moreatharvaa
Copy link

already familiar with the language and would love to contribute!

@jainendra001
Copy link

jainendra001 commented Jun 4, 2023 •
edited

Hi @invinayak007 ,

I'm Jainendra Tripathy, and I am interested in contributing to this project. I'm an AI and ML developer with good hands-on experience in Python.
I have a strong understanding of Python, Data validation techniques., and I'm always looking for new ways to learn and grow. I'm also a quick learner and always willing to help.
I'm available to contribute . I'm eager to get started and make a difference. I'm confident I can help you take this project to the next level.
If you're interested in having me contribute, please let me know. I'm happy to answer any questions you have.
And I am also applying to the C4GT too please consider my application.

Thanks,
Jainendra Tripathy

@ChakshuGautam ChakshuGautam changed the title File Validator for cQube [C4GT] File Validator for cQube Jun 5, 2023
@invinayak007
Copy link
Collaborator Author

As many of you were asking about more clarity, I am adding few things here

  1. End user will run the utility (possibly a flask application or you can come up with other ideas)
  2. User will select the particular program(which cQube supports) for which the file needs to be validated.
  3. Based on the program selected, the utility should fetch the specification/schema which is stored in DB (in JSON).
  4. Specific validations as per the specification/schema and other generic validations as discussed in the issue description should happen.
  5. You can think of some more UI components to help user define custom validations and process (ex: user can enter column name and range for the values)
  6. You can fine more about schemas here
  7. Finally it should prepare a report indicating the validation issues.

Let me know if you have any further questions on this.

@VivekPrakashNitH
Copy link

I am interested in this project as i very familiar with this language and has built many ML related projects on this . Link to one of my end project :-
https://github.com/VivekPrakashNitH/open-air-gym
and many more ML and AI related projects like this

@AdityaSen-1606
Copy link

Hi @invinayak007, I am interested in solving this issue. I have the skills to solve this issue and am confident that I will add significant value to this project.

@VivekPrakashNitH
Copy link

VivekPrakashNitH commented Jun 10, 2023 via email

@Fayouzz
Copy link

Fayouzz commented Jun 12, 2023

Dear Mentor,
I have submitted my proposal for "[C4GT] File Validator for cQube #1". Looking forward to learning and contributing :)
Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests