Such activity has been illegal since 2010, but until now had not invoked the penalties.
It follows a lobbying campaign by country's music industry.
But critics said that efforts should have remained focused on stopping users making such material available.
In Japan illegal uploads of copyright infringing music and videos carry a maximum 10 year prison sentence and a 10 million yen fine.
Sales figures suggest the country is the world's second-largest music market after the US.
Photo Left: People wearing Anonymous masks staged a protest against the change in law earlier this year
In theory the new download punishments can be enforced if a user is found to have copied a single pirated file.
The Recording Industry Association of Japan had pushed for the move, suggesting that illegal media downloads outnumbered legal ones by about a factor of 10.
The figure is based on a 2010 study which suggested that people in the country downloaded about 4.36 billion illegally pirated music and video files and 440 million purchased ones that year.
It added that the disparity was likely to have increased over the following months.
"This revision will reduce the spread of copyright infringement activities on the internet," said the body's chairman Naoki Kitagawa, who is also chief executive of Sony Music Entertainment Japan, earlier this year.
Politicians voted through the change in June.
Shortly afterwards the website of the government's finance ministry was defaced and the sites of the Supreme Court, the DPJ and LDP political parties, and the Japanese Society for Rights of Authors, Composers and Publishers were briefly taken offline after cyber attacks.
The following month a group of masked activists associated wearing masks associated with the Anonymous hacktivist movement staged a protest in Tokyo.
About 80 participants picked up rubbish from the ground in the city's Shibuya shopping district for an hour to publicise their opposition to the plan.
The Japan Federation of Bar Associations, a group representing legal professionals, also issued a statement saying the offence should have remained a civil, rather than criminal, matter.
"Treating personal activities with criminal punishments must be done very cautiously, and the property damage caused by individual illegal downloads by private individuals is highly insignificant," it said.
However, the efforts did not sway the politicians.
Japan's action is part of a wider international crackdown on online piracy.
Over recent months the US has taken the digital locker service Megaupload offline; Ukraine has shut down the BitTorrent site Demonoid; the UK has jailed the owner of the Surfthechannel video link provider; and several countries have restricted access to The Pirate Bay torrent service - the founder of which was recently deported from Cambodia to Sweden to face tax charges.
France also recently fined one of its citizens for the first time under its "three strikes" rule which allows it to impose a fine if a suspected pirate ignores three warnings about their activity.
However, attempts to introduce new laws have run into problems elsewhere.
The US put off votes on Sopa (Stop Online Piracy Act) and Pipa (Protect IP Act) in January after Wikipedia and thousands of other sites staged blackouts in protest.
The European Parliament also voted to reject Acta (Anti-Counterfeiting Trade Agreement) in July after opposition across the continent.