According to the manual
Supported subtitle format(external) :*.smi, *.srt, *.sub(MicroDVD, SubView1.0/2.0), *.ass, *ssa, *.txt(TMPlayer),*.psb(PowerDivX)y
Supported subtitle format(internal) :XSUB(Only, XSUB is the subtitle for-mat used in DivX files), Text UTF- 8,SRT,ASS,SSA(Only, this is the subtitleformat used in DivX-Plus HD files)
Personally I just try to stick with .SRT format subtitles because my PVR and my LG TV can use them and they are just text files which can be edited. For mkv files these can be external files with the same name as the video but with a .SRT extension or they can be inside the MKV file.
If you can't get your .SUB to work and it can't be converted, just try searching for a .SRT version.