I begin with an elementary remark. Let $T\supset L$ be tori with $T$ acting on a finite dimensional vector space $V$. Let $\chi_{1},\ldots,\chi_{n}$ be the characters of $T$ occurring in $V$. Then $T$ acts faithfully on $V$ if and only if $\chi_{1},\ldots,\chi_{n}$ span $X^{\ast}(T)$ as a $\mathbb{Z}{}% $-module --- assume this. The characters of $T$ occurring in $\mathrm{End}(V)$ are $\{\chi_{i}-\chi_{j}\}$, and the set of those occurring in $\mathrm{End}(V)^{L}$ is \begin{equation} \{\chi_{i}-\chi_{j}\mid\chi_{i}|L=\chi_{j}|L\}. \tag{*}% \end{equation} On the other hand, \begin{equation} X^{\ast}(T/L)=\{\sum a_{i}\chi_{i}\mid\sum a_{i}\chi_{i}|L=0\}. \tag{**} \end{equation} Thus, $T/L$ will act faithfully on $\mathrm{End}(V)^{L}$ if the set (*) spans the $\mathbb{Z}{}$-module (**).
I now prove the statement. With the notations of Milne 1999b, Section 6 (especially p22,23 of the pdf file on my site), $T^{\Psi}$ acts on a realization of $h_{1}A^{\Psi}$ through the characters $\psi_{0},\ldots,\psi_{n-1},\iota \psi_{0},\ldots,\iota\psi_{n-1}$, where the $\psi_{i}$ have been numbered so that $\pi(\psi_{0})=\cdots=\pi(\psi_{d-1})=\pi_{0}$, $\pi(\psi_{d})=\cdots =\pi(\psi_{2d-1})=\pi_{1},$ etc.. Now $\sum a_{i}\cdot\psi_{i}|L^{\Psi}=\sum a_{i}\cdot\pi(\psi_{i})$, which is zero if and only if $\sum_{i=0}^{d-1} a_{i}=0$, $\sum_{i=d}^{2d-1} a_{i}=0$, … ; but then $$ \sum a_{i}\psi_{i}=\sum_{i=0}^{d-1}a_{i}(\psi_{i}-\psi_{0})+\cdots, $$ which (by the remark) shows that $T^{\Psi}/L^{\Pi}$ acts faithfully on $\underline{\mathrm{End}}(h_{1}A^{\Psi})^{L^{\Pi}}$. It is even easier to show that $T^{\bar{\Psi}}/L^{\bar{\Pi}}$ acts faithfully on $\underline{\mathrm{End}}(h_{1}A^{\bar{\Psi}})^{L^{\bar{\Pi}}}$ and deduce that $T^{A^{\Psi}\times A^{\bar{\Psi}}}/L^{A^{\Pi}\times A^{\bar{\Pi}}}$ acts faithfully on $\underline{\mathrm{End}}(h_{1}(A^{\Psi}\times A^{\bar{\Psi}% }))^{L^{A^{\Pi}\times A^{\bar{\Pi}}}}$. As \[ P^{K}/L^{K}\hookrightarrow T^{A^{\Psi}\times A^{\bar{\Psi}}}/L^{A^{\Pi}\times A^{\bar{\Pi}}}% \] (cf. ibid. Lemma 6.9), this implies the statement.$\square$