Start networking and exchanging professional insights

Register now or log in to join your professional community.

Follow

What is the easiest way to remove the data redundancy via using SQL?

we have single table with lot of multiple records and we like to remove the data redundancy from the table. the logic is we want to remove one record the previous record from the table by using sql

user-image
Question added by Tanvir Khan , Collection strategy Analytics , Abu Dhabi Islamic Bank
Date Posted: 2013/06/22
Gayasuddin Mohammed
by Gayasuddin Mohammed , Advocate , Practicing Law before High Court at Hyderabad

select distinct f1,f2,....fn from table - will get you distinct record in the table

best use the cursor with the above querry and store the records by inserting in a similar new table by having a primary key on it by iterating the cursor and once it is done, you can delete your old table.

your job is done.

Farah Azar
by Farah Azar , IT Project Manager , MetLife

Select Distinct ..............

Imeely Carlos
by Imeely Carlos , Business Application Analyst and Developer , LeasePlan Emirates LLC

Select distinct field from table...

Rizwan Afgan
by Rizwan Afgan , SQL Server Administrator / Developer , Oracas (Pvt) Ltd

First of all implement at least third normal form. Second use SELECT DISTINCT. That's all!

haytham Hamdy
by haytham Hamdy , Developer , SAP

Select distinct Column_Name From Table_Name

Jamal Hamed
by Jamal Hamed , IT Manager / Team Leader , BISC Solutions

your Database must be  normalized.

Kashif Hafeez Sharepoint Engineer BPM
by Kashif Hafeez Sharepoint Engineer BPM , Software Engineer Sharepoint/Support/BPM , FGB (First Gulf Bank)

Marwan answer is right as there is only one approach that is fast to  achieve this task-

keep data in temp table and delete constraints that are making data redundant from previous table and take data back to previous table.

 

Mohammad Masadeh
by Mohammad Masadeh , SharePoint Lead Developer , Port Customs And Freezone Corporation Dubai

Yea it's just select distinct 

Deleted user
by Deleted user

Data normalisation is what you are looking for...

Normalisation is the process of taking data from a problem and reducing it to a set of relations while ensuring data integrity and eliminating data redundancy

  • Data integrity - all of the data in the database are consistent, and satisfy all integrity constraints.
  • Data redundancy – if data in the database can be found in two different locations (direct redundancy) or if data can be calculated from other data items (indirect redundancy) then the data is said to contain redundancy.

1st Normal Form:

  • A relation is in1NF if it contains no repeating groups
  • To convert an unnormalised relation to1NF either:
  • Flatten the table and change the primary key, or
  • Decompose the relation into smaller relations, one for the repeating groups and one for the non-repeating groups.
  • Remember to put the primary key from the original relation into both new relations.
  • This option is liable to give the best results.

2nd Normal Form:

  • A relation is in2NF if it contains no repeating groups and no partial key functional dependencies
  • Rule: A relation in1NF with a single key field must be in2NF
  • To convert a relation with partial functional dependencies to2NF. create a set of new relations:
  • One relation for the attributes that are fully dependent upon the key.
  • One relation for each part of the key that has partially dependent attributes

3rd Normal Form:

  • A relation is in3NF if it contains no repeating groups, no partial functional dependencies, and no transitive functional dependencies
  • To convert a relation with transitive functional dependencies to3NF, remove the attributes involved in the transitive dependency and put them in a new relation
  • Rule: A relation in2NF with only one non-key attribute must be in3NF
  • In a normalised relation a non-key field must provide a fact about the key, the whole key and nothing but the key.
  • Relations in3NF are sufficient for most practical database design problems. However,3NF does not guarantee that all anomalies have been removed.

Deleted user
by Deleted user

Better to reorganize your DB.
Better structure of your table.
Know how to create relations between tables.
- Relations somehow is the key for this, better organized relations, less Redundancy

Marwan Al Haj Kassem
by Marwan Al Haj Kassem , ِAnalyst Programmer , UNRWA

To do that you can follow the following steps :1.
create new table with same structure , its very easy to use script create changing only the name of table.2.
insert into new table distinct records.
insert into new table values select (distinct column( key),columns,....) from original table ;3.
drop the records of original table , delete * from original table.
( you can drop the table and create again with drop and create scripts to clear indexes and incremental values.
)4.Insert record on the new table back to the original .
like insert into original tables values select columns from newtable.
This is a way .
another professional way you can count the appearance of record.
select key,count(key) from table group by key having count(key)>1 delete which appear more than one limiting the delete to1 on every delete till you have no more records appearing more the one .
if you need more help on the second way i can give you real example.
Regards.

More Questions Like This