I've had the issue as well a long while ago (so I don't remember exactly how I fixed it), but you did have the right idea at the start. It's to do with how the video is encoded then transferred. Even files downloaded which were encoded with the correct codecs failed and required re-encoding. Wikipedia has the supported video and audio codecs, I'd recommend you re-encode and try to force the resolution (find this info here
On to the transfer, I noticed (when I was using this mode) that transferring video while the device was mounted as a Windows Media Player device also caused issues, because windows tried to re-encode the video on the fly and screwed up a working file. I've had successes getting videos played after transferring them with the Archos mounted as a PC Hard Drive, provided the files can be played back on PC and are encoded correctly.
With all this said, I no longer use my Archos for video playback; the battery life is just not there anymore, so I won't be able to assist too much. However, from one file that did work, this is the codec info from VLC (video is 50 mins long, about 700 MB, and an avi). I hope it helps.
Codec: MPEG-4 Video (XVID)
Frame rate: 29.970030
Codec: A52 Audio (aka AC3) (a52 )
Sample rate: 48000 Hz
Bitrate: 192 kb/s
EDIT: If you feel your Archos drive is close to death, you can still extend the life of the unit; you can replace the drive if you wish to try that. There's guides on the forums (I'm 100% sure they're there) as well as on the g3nius
page and a how-to page on google
. I personally don't use the stock regular 120GB drive anymore as it's too small, instead opting for a 1TB drive.