Now from limma output, I found genes with negative log fold change that are express higher in the insensitive samples and genes with positive log fold changes that are express higher in the sensitive samples, but, what I want to know is how can I find down regulated genes in each phenotype?
What do you mean by "down regulated genes in each phenotype"? By its nature, a DE analysis looks for changes in expression between two or more conditions. In your case, this necessarily involves comparing phenotypes against each other. I don't know what else you're expecting to get from limma.
Also, your filter doesn't seem to make much sense. You're always throwing away half of your probes, regardless of how highly or lowly expressed they might be. You should get a better idea of the filter threshold from your control probes - have a look at Section 17.4 of the user's guide, for example.