As a writer of science fiction and fantasy, and on behalf of all the variations and sub-genres such as urban fantasy, alternate history and steampunk which collectively make up "speculative fiction," I'd argue that genre fiction is different from literary fiction.
Whether it's dealing with ray guns and rocket ships, swords, sorcery or fur and fangbangers, speculative fiction's unifying, identifying characteristic is that it doesn't attempt to mimic real life in the way that literary fiction does. It stands apart from the world we know. It takes us away to an entirely secondary realm, be that Middle Earth or Westeros, or to an alternate present where vampires and werewolves really do exist and you ring 666 to report a supernatural crime…
Speculative fiction can be considerably harder to write than literary fiction…When readers are paying close attention to every hint and clue, the writer needs to have internal logic, consistency of character and scene-setting absolutely nailed down. Readers have to be convinced that this unfamiliar world is solidly real if they're ever going to suspend disbelief and accept the unreal, whether that's magic and dragons or faster-than-light travel.
Juliet McKenna, The Guardian, April 18, 2014