The court ruling was the latest to sharply criticize a major initiative by Chancellor Angela Merkel's government and one of the strongest steps yet defending citizen rights from post-Sept. 11 terror-fighting measures.
The ruling comes amid a European-wide attempt to set limits on the digital sphere, that includes disputes with Google Inc. over photographing citizens for its Street View maps.
The Karlsruhe-based Federal Constitutional Court ruled that the law violated Germans' constitutional right to private correspondence and failed to balance privacy rights against the need to provide security. It did not, however, rule out data retention in principle.
The law had ordered that all data - except content - from phone calls and e-mail exchanges be retained for six months for possible use by criminal authorities, who could probe who contacted whom, from where and for how long.
"The disputed instructions neither provided a sufficient level of data security, nor sufficiently limited the possible uses of the data," the court said, adding that "such retention represents an especially grave intrusion."
The court said because citizens did not notice the data was being retained it caused "a vague and threatening sense of being watched."
Nearly 35,000 Germans had appealed to the court to overturn the law, which stems from a 2006 European Union anti-terrorism directive requiring telecommunications companies to retain phone data and Internet logs for a minimum of six months in case they are needed for criminal investigations.
Civil rights groups had fiercely opposed the law, arguing that even excluding the content of phone calls and e-mails could allow authorities too deep a view into their personal sphere.
"Massive amounts of data about German citizens who pose no threat and are not suspects is being retained," Germany's commissioner for data security issues, Peter Schaar, told ARD television.
Security experts argued the information is crucial to being able to trace crimes involving heavy use of the Internet, including tracking terror networks and child pornography rings.
While the court upheld the EU directive as necessary to fight terror, it took issue with how the German law had interpreted it and ordered further restrictions on access to the data.
Changes ordered by the court included granting access to the data only by court order and only in the event of "concrete and imminent danger." The court further insisted the information be stored in the private sector so it was not concentrated in one spot.
Germans, in particular, are sensitive to privacy issues, based on their experiences under the Nazis as well as the former East Germany's Communist dictatorships, where information on individuals was collected and abused by the state.