How To Remove This Special Character?

I was trying to unify the lines in my file when I observed the following: word1 word2 word1 word2 I did not understand why these lines were not combined so I opened the file in vim

Solution 1:

U+FEFF is the Byte Order Mark character, which should only occur at the start of a document. In documents, it should be treated as a ZERO WIDTH NON-BREAKING SPACE. If this causes issues, you can remove it like any other character:

>>> s = u'word1 \ufeffword2'>>> s = s.replace(u'\ufeff', '')
>>> s
u'word1 word2'

_{(In Python 3.1 or 3.2, drop the u in front of strings)}

Solution 2:

Have you tried mytext.split(string.whitespace) ?