FAT is not as secure or robust as NTFS - but that doesn't mean secure in the sense of malware/hacking/viruses/etc. It means that FAT filesystems are more susceptible to, and less tolerant of failures, and cannot recover themselves as easily.
In practice FAT is not really bad - if it was people would be having problems all over the place, and they aren't.
You are sensible to hesitate about conversion. If you decided to do it, you should definitely make a complete and comprehensive backup first. Whilst the conversion goes right nearly all the time - it wouldn't be nice to be one of the failures.