maybe just check TERM? although to be honest--do you even need a pager if your shell/terminal is nice enough?

I rarely pipe to a pager because kitty/fish/tmux support OSC 133 pretty well and I can press one button to go up to the previous command prompt and another button to go back. I can press a few keys to search the scrollback. It all works seamlessly across SSH sessions too if you set it up right.

    # tmux
    bind-key -T copy-mode C-Home send -X history-top
    bind-key -T copy-mode C-End  send -X history-bottom
    bind -T copy-mode C-Up send-keys -X previous-prompt
    bind -T copy-mode C-Down send-keys -X next-prompt

    # kitty
    map ctrl+up scroll_to_prompt -1
    map ctrl+down scroll_to_prompt 1
    map shift+PAGE_UP scroll_page_up
    map shift+PAGE_DOWN scroll_page_down

    # fish shell config.fish
    if test -z "$SSH_TTY"; and test -z "$TMUX"
        bind pageup "kitty @ scroll-window 1p-"
        bind pagedown "kitty @ scroll-window 1p"
    end