I have two independent apps:
Both apps authenticate user against Azure Active Directory using oauth2 flow.
I'd like to use the Access Token of second app and use it to access the API of first app.
How can I do it so oauth2_proxy validates it?
The solution was to add app's audience of second application to --extra-jwt-issuers argument. Here is the configuration and nitty-gritty details of how to use it. 
- --azure-tenant=11111111-2222-3333-4444-55555555
- --email-domain=*
- --http-address=0.0.0.0:4180
- --provider=oidc
- --set-authorization-header=true
- --set-xauthrequest=true
- --pass-access-token=true
- --pass-authorization-header=true
- --pass-user-headers=true
- --pass-host-header=true
- --skip-jwt-bearer-tokens=true
- --oidc-email-claim=oid
- --oidc-issuer-url=https://sts.windows.net/11111111-2222-3333-4444-55555555/
- --extra-jwt-issuers=https://sts.windows.net/11111111-2222-3333-4444-55555555/=api://app1-2222-3333-4444-55555555,https://sts.windows.net/11111111-2222-3333-4444-55555555/=api://app2-2222-3333-4444-55555555As you can see above I'm actually adding two issuers, this is because I'm using two apps that need to be authenticated.