Broadly, a set of stories which are similar in some ways. A genre convention is anything which that set of stories have in common. This is a tricky term in use, however.

References:

• GNS and Other Matters of Role-playing Theory
• Understanding Genre in Roleplaying