then you will find that it reads the file as you expect.
You're getting confused at the moment (1) because the Chromosome number has been converted into a factor and (2) because I suspect your file doesn't actually have a line of column headings.
Actually read.delim() has converted my_data[1,1] correctly. You will notice that "13" is the 5th possible value that my_data[,1] can take on according to the list of Levels. When you coerce my_data[1,1] to numeric (why did you do that?), it records which level the element is in terms of the ordered list of levels (5th level) not the actual level ("13"). That's why you get d=5. If you wanted the actual level, you would use as.character(my_data[1,1]) instead. Note that Chromosome number is best stored as character rather than numeric because "X" and "Y" are possible values.