I'm using React with Tanstack Query and want to consume a streaming API endpoint.

I basically retrieve subtrees representing table rows but the amount of data is huge so I can't store all the data inside a state array ( I think ). The data might reach ~1GB and some people might crash their browser tab. Further React would keep rerendering until the streaming is done so the app would keep stuttering. I know Tanstack Query provides an experimental streamedQuery hook but as mentioned before it would store all the data into a big state array.

I think for the printing mode, when loading all the data, the only solution would be to create my own hook processing the streamed data and directly appending the row component to the DOM to save memory and performance.

I tried to create my own hook for this ( please let me know if React / Tanstack Query or other popular libraries can help with that )

function useAllItemsNDJSONStreamToDOM() {
    const tableBodyRef = useRef(null);
    const [isPending, setIsPending] = useState(true);
    const [error, setError] = useState(null);

    useEffect(() => {
        const controller = new AbortController();
        const tableBody = tableBodyRef.current;

        if (!tableBody) {
            setIsPending(false);
            return;
        }

        tableBody.innerHTML = "";

        async function run() {
            try {
                const response = await fetch("/all-items", {
                    signal: controller.signal,
                });

                if (!response.ok) {
                    const { status, statusText } = response;

                    throw new Error(
                        JSON.stringify({
                            status,
                            statusText,
                        }),
                    );
                }

                if (!response.body) {
                    throw new Error("response body is null");
                }

                const reader = response.body.getReader();
                const decoder = new TextDecoder();

                let buffer = "";

                while (true) {
                    const { done, value } = await reader.read();

                    if (done) {
                        break;
                    }

                    buffer += decoder.decode(value, { stream: true });

                    const lines = buffer.split("\n");

                    buffer = lines.pop() ?? "";

                    for (const line of lines) {
                        if (!line.trim()) {
                            continue;
                        }

                        try {
                            const parsedJSON = JSON.parse(line);
                            const items =
                                itemsSchema.parse(parsedJSON);

                            const tableRow = document.createElement("tr");
                            const root = ReactDOM.createRoot(tableRow);

                            root.render(
                                /* 
                
                  TODO

                  Row component
                
                */
                

                  
Streamed row:
                  
{JSON.stringify(items, null, 2)}
                
                            );

                            tableBody?.appendChild(tableRow);
                        } catch (error) {
                            console.error(error);
                        }
                    }
                }
            } catch (err) {
                if (!controller.signal.aborted) {
                    setError(err);

                    if (tableBody) {
                        tableBody.innerHTML = "";
                    }
                }
            } finally {
                setIsPending(false);
            }
        }

    setError(null);
        setIsPending(true);
        run();

        return () => {
            controller.abort();
        };
    }, []);

    return {
        tableBodyRef,
        isPending,
        error,
        hasError: !!error,
    };
}

export { useAllItemsNDJSONStreamToDOM };

The consuming component currently looks like

export default function PrintSpreadSheet() {
    const { tableBodyRef, isPending, error, hasError } =
        useAllItemsNDJSONStreamToDOM();

    return (
        

            {hasError && 
{JSON.stringify(error)}
}
            {isPending ? "Still streaming rows..." : "Finished streaming rows!"}
            
        
    );
}

I think the hook almost works as expected but the state `isPending` is always truthy. So the component instantly renders

Finished streaming rows!

although the API keeps streaming and I would expect

Still streaming rows...

Do you know how to fix ( + improve ) the hook? Or maybe you know better alternative solutions for this.