What is the right way to specify accent colors for high contrast mode ("Increase Contrast" in settings)?

When I specify accent colors in my asset file, and further specify the high contrast variants, it does use those colors, but then SwiftUI further 'increases' contrast, but for some reason it assumes my color is a foreground color, which means that it decreases the actual contrast, because I'm using the color as a background, not foreground.

import SwiftUI
 
@main
struct HighContrastBlueDemoApp: App {
    var body: some Scene {
        WindowGroup { ContentView() }
    }
}
 
struct ContentView: View {
    var body: some View {
        VStack(spacing: 0) {
            Color.accentColor
                .overlay {
                    Text("Color.accentColor — NOT in NavigationStack")
                        .foregroundStyle(.white)
                }
 
            NavigationStack {
                Color.accentColor
                    .overlay {
                        Text("Color.accentColor — inside NavigationStack")
                            .foregroundStyle(.white)
                    }
                    .toolbar(.hidden, for: .navigationBar)
            }
        }
        .ignoresSafeArea()
    }
}
 
#Preview { ContentView() }

Screenshot shows where it is most annoying me: in dark mode in iOS 18.5, but the same issue happens in 26.4 (don't know 🙈 about 26.5) and in light mode.

So... is there a 'right' way to do this/work around it? Like, if this goes inside a super complex SwiftUI view like a Form, it's very difficult to manually specify the icon colors without breaking the formatting. Specifying the color via sRBG values is not workable. Do people just ¯\_(ツ)_/¯ and let Increase Contrast make their apps look random?