When I use your very small gene list (test) gseKEGG is working fine; no unexpected errors are reported, except that no enriched gene sets are found when using minGSSize=5 (which of course is to be expected with such very small input list.

The type/structure of your gene list (Named num) is the same as one of my lists, so that should be fine as well.

My suggestions: lower minGSSize to e.g. 5 or 10 (or even 1), and increase pvalueCutoff  to 0.9 or 1. Although the min gene set size and p-value cutoff make no sense now, this will / should allow gene sets to be returned, confirming the analysis as such is working, You can then fine-tune these settings to your need. I always set nPerm to 10,000. Please note that using your very small input list with minGSSize=1 indeed gives (nonsense) results.

Also be sure to use the latest version of clusterProfiler/DOSE.

Finally, check whether gseKEGG runs with the sample dataset. If so, the the problem is specific to your input list.

minGSSize defines the minimum size of the KEGG gene set to be included.Gene sets smaller than this are excluded from the analysis. If you set it to low, the possibility of obtaining inflated scores is high. Remember: "...the GSEA normalization is not very accurate for extremely small or extremely large gene sets. For example, for gene sets with fewer than 10 genes, just 2 or 3 genes can generate significant results. Therefore, by default, GSEA ignores gene sets that contain fewer than 25 genes or more than 500 genes; defaults that are appropriate for datasets with 10,000 to 20,000 features. However, keep in mind the possibility of inflated scorings for very small gene sets and inaccurate normalization for large ones." Source. I usually set minGSSize to 15, and maxGSSize to 500.

Using your very limited list (which basically shows data format/input of your list is OK).

test <- c(8.43, 8.35, 7.71, 7.18)
names(test) <- c("15496", "28248", "100039008", "28248")

> AdvsDDSLiverGSEA <- gseKEGG(geneList =test,
+ organism     = 'mmu',
+ nPerm        = 1000,
+ minGSSize    = 5,
+ pvalueCutoff = 1,
+ verbose      = FALSE)
no term enriched under specific pvalueCutoff...
>

Now setting minGSSize to 1.

> AdvsDDSLiverGSEA <- gseKEGG(geneList =test,
+ organism     = 'mmu',
+ nPerm        = 10000,
+ minGSSize    = 1,
+ pvalueCutoff = 1,
+ verbose      = FALSE)
> AdvsDDSLiverGSEA
#
# Gene Set Enrichment Analysis
#...@geneList    Named num [1:4] 8.43 8.35 7.71 7.18
 - attr(*, "names")= chr [1:4] "15496" "28248" "100039008" "28248"
#...nPerm        10000
#...pvalues adjusted by 'BH' with cutoff <1
#...5 enriched terms found
'data.frame':   5 obs. of  11 variables:
 $ ID             : chr  "mmu00140" "mmu01100" "mmu04913" "mmu04925" ...
 $ Description    : chr  "Steroid hormone biosynthesis" "Metabolic pathways" "Ovarian steroidogenesis" "Aldosterone synthesis and secretion" ...
 $ setSize        : int  1 1 1 1 1
 $ enrichmentScore: num  1 1 1 1 0.667
 $ NES            : num  1.197 1.197 1.197 1.197 0.798
 $ pvalue         : num  0.506 0.506 0.506 0.506 0.506
 $ p.adjust       : num  0.506 0.506 0.506 0.506 0.506
 $ qvalues        : num  0.506 0.506 0.506 0.506 0.506
 $ rank           : int  4 4 4 4 4
 $ leading_edge   : chr  "tags=100%, list=100%, 
<<snip>>
>

gseKEGG using sample data set:

> data(geneList)
> test.run <- gseKEGG(geneList = geneList,
+ organism     = 'hsa',
+ nPerm        = 10000,
+ minGSSize    = 15,
+ pvalueCutoff = 0.05,
+ verbose      = FALSE)
> test.run
#
# Gene Set Enrichment Analysis
#
#...@organism    hsa
#...@setType     KEGG
#...@geneList    Named num [1:12495] 4.57 4.51 4.42 4.14 3.88 ...
 - attr(*, "names")= chr [1:12495] "4312" "8318" "10874" "55143" ...
#...nPerm        10000
#...pvalues adjusted by 'BH' with cutoff <0.05
#...45 enriched terms found
'data.frame':   45 obs. of  11 variables:
 $ ID             : chr  "hsa04510" "hsa03030" "hsa03050" "hsa03008" ...
 $ Description    : chr  "Focal adhesion" "DNA replication" "Proteasome" 
<<snip>>
..