The process of Cleaning and Verifying Data is the crucial quality control stage that occurs after data has been accessed and collated. In the context of TESDA's National Competency Assessment, this step ensures that the information used for certification is accurate, consistent, and reliable.

"Cleaning" involves identifying and correcting errors and inconsistencies in the data, while "Verifying" is the act of confirming that the data accurately reflects the original source and the candidate's actual performance. This process is fundamental to maintaining the integrity and credibility of every National Certificate (NC) or Certificate of Competency (CoC) issued.

No dataset is perfect. The final preparation step is to detect and correct errors and inconsistencies.

In reality, data is rarely perfect. "Data Errors" also known as data quality issues or dirty data, are inaccuracies and inconsistencies within a dataset that can significantly undermine the reliability of any analysis or decision-making process based on that data. Understanding the various types of data errors is the first critical step toward identifying, correcting, and ultimately preventing them.

Understanding common errors is the first step to cleaning them.

Data cleaning is a critical step in the data analysis pipeline, ensuring datasets are accurate, consistent, and reliable for informed decision-making. It involves identifying, correcting, or removing errors, duplicates, and inconsistencies

Cleaning Techniques:

Reviewing: This is the initial exploratory phase where you visually inspect the dataset row by row or column by column. It helps detect outliers, formatting issues (e.g., mixed date formats), or logical errors (e.g., negative ages). While time-intensive for large datasets, it's invaluable for small-scale or qualitative data.

Sorting & Filtering: Sorting arranges data alphabetically or numerically (e.g., by name or date), grouping anomalies for easy identification. Filtering then isolates subsets (e.g., rows with specific keywords) to drill down on problems like duplicates or outliers.

Cross-Validation: Compare the collated data against the original source documents to verify accuracy. This verification step involves reconciling cleaned data with primary sources (e.g., scanning original document or databases) to confirm entries. It's especially useful for ensuring no transcription errors occurred during data entry.

Secure Storage and Sign-Off: Once the data is cleaned and verified, it must be stored securely. This means saving it in the correct, access-controlled location. The final step is to obtain a sign-off/clearance from your supervisor, which confirms that the data has been prepared in accordance with the required standards.