How to archive old history data in dynamics crm 3.0 database due to decrease its size ?

There are multiple solutions for this, you can do it either by:

1- Creating a new organization on the same deployment manager and migrate the records from one instance to the new one deleting the records from the original organization, but to access the historical data you need to access the new organization

2- Deactivate the old idle unwanted records and improve the indexes, also delete the closed system jobs to reduce the DB size of the organization