PyScript 💻 - ╰╸pyscript-bugs

Hello bugs

1

Trying to modify the py-onclick attribute to call a different function after it has been clicked once.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width,initial-scale=1" />
    <script defer src="https://pyscript.net/latest/pyscript.js"></script>
    <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.1.3/dist/css/bootstrap.min.css" rel="stylesheet">
    <py-script src="bug.py"></py-script>
</head>



<body>
    <div class="container-fluid">
        <div class="row justify-content-center">
            <div class="col-2">
                <input id="next1" type="submit" class="p-4 btn btn-primary" value="click me" py-onclick="test()">
            </div>
        </div>
    </div>
</body>

import js
from js import document

def test():
    button = document.getElementById("next1")
    console.log("this is test 1")
    console.log(button.getAttributeNames())
    console.log(button.getAttribute("py-onclick"))
    button.removeAttribute("py-onclick")
    console.log(button.getAttributeNames())
    console.log(button.getAttribute("py-onclick"))
    button.setAttribute("py-onclick", "test2()")
    console.log(button.getAttributeNames())
    console.log(button.getAttribute("py-onclick"))


def test2():
    console.log("success! this is test 2")
    document.getElementById("next1").setAttribute("py-onclick", "test()")

so, the attribute value does get changed to reference the second function, but when clicking the button, it only performs the initial function it originally had. console log shows two clicks of the button (edited)

I haven't look much into this yet, but the py-onclick and py-click have event listeners attached to them when the app starts, I don't think simply removing the attribute and adding it again will achieve what you want. If you want to have a different behaviou on different clicks, perhaps you could add a variable with a click count, then depending on the count your test() function would trigger one or the other flow

Ah okay, that makes sense ! - thanks for the suggestion, sounds like a great solution - I'll give it a go

update: adding a counter achieved the desired interactions, thank you Fabio ! (edited)

awesome!

When inserting HTML using the "innerHTML =" syntax the py-onclick attribute is not recognized. Using the regular onclick here works as expected.

1

interesting, thanks for reporting @tuxprint! could you create a bug on gh so it doesn't get lost?

11:08

maybe with a small reproducible ex. would be great

3l3ktr4

interesting, thanks for reporting @tuxprint! could you create a bug on gh so it doesn't get lost?

I posted an issue to the github. I included a simple HTML document that illustrates the issue

1

awesome!

thanks @tuxprint I'll give it a look

hello, anyone there? I am getting error while pyscript is setting python environment. Seems like issue is with panel package.

<py-config type="json">
      { "terminal": false, "packages": ["altair","bokeh" , "numpy", "pandas",
      "scikit-learn", "panel==0.13.1"] }
    </py-config>

exceptions.ts:52 (PY1001): Unable to install package(s) 'altair,bokeh,numpy,pandas,scikit-learn,panel==0.13.1'. Reason: Can't find a pure Python 3 Wheel for package(s) 'altair,bokeh,numpy,pandas,scikit-learn,panel==0.13.1'. See: https://pyodide.org/en/stable/usage/faq.html#micropip-can-t-find-a-pure-python-wheel for more information.

@hood I think this is just like the error I've pinged you?

12:24

I also reproduced this "cant find a pure python wheel" with packages other than numpy

12:24

did you make progress there? (sorry I've been out for a week, still catching up on email)

3l3ktr4

did you make progress there? (sorry I've been out for a week, still catching up on email)

resolved it by downgrading to 2022.12.1 i.e. <script defer src="https://pyscript.net/releases/2022.12.1/pyscript.js"></script>

the problem is from Panel --> Bokeh --> tornado and tornado is a async server library which can't be used in Pyodide: ValueError: Can't find a pure Python 3 wheel for 'tornado>=5.1'. The version of Bokeh distributed with Pyodide has all the webserver stuff removed, including the tornado dependency. So no other version can be used. Pyodide 0.22 comes with Bokeh v3.0.3, but Panel < 1.0 has a version cap for Bokeh < 2.5.0 and the Panel 1.0.0 alpha releases have a version cap for Bokeh > 3.1.0. So no version of Panel can be used in Pyodide 0.22 unless you get a custom version of Bokeh

20:53

@philippjfr

20:53

perhaps we could release Pyodide 0.22.2 with a fix for this but we probably don't care enough

Would suggest using the custom wheels we distribute on our CDN for the time being. We used that approach for the examples on PyScript.net

21:02

https://cdn.holoviz.org/panel/0.14.3/dist/wheels/bokeh-2.4.3-py3-none-any.whl https://cdn.holoviz.org/panel/0.14.3/dist/wheels/panel-0.14.3-py3-none-any.whl

21:03

Panel 1.0 is about two to three weeks out, at which point making Bokeh 3.1 available in pyodide directly would be great. (edited)

some day it would be nice to separate the Pyodide package distro from libpyodide / the ffi

21:06

it's totally possible now to use Pyodide 0.22 with a lock file that points to the Pyodide 0.21 package distro or vice versa

This example from the PyScript.com Getting Started collection is not working:

nascif

This example from the PyScript.com Getting Started collection is not working:

https://6fd75ce3-df0a-4384-b255-195e41c4f02f.pyscriptapps.com/0201e04e-18e2-42f7-84ed-8760af05c129/latest/ "Service Worker Example"

nascif

https://6fd75ce3-df0a-4384-b255-195e41c4f02f.pyscriptapps.com/0201e04e-18e2-42f7-84ed-8760af05c129/latest/ "Service Worker Example"

On the Edit UI, you can see the following error on the console: Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user’s experience. For more help http://xhr.spec.whatwg.org/ utils.ts:64:12 Uncaught (in promise) TypeError: ServiceWorker script at https://pyscript.com/api/content/6fd75ce3-df0a-4384-b255-195e41c4f02f/0201e04e-18e2-42f7-84ed-8760af05c129/latest/piodide-sw.js for scope https://pyscript.com/api/content/6fd75ce3-df0a-4384-b255-195e41c4f02f/0201e04e-18e2-42f7-84ed-8760af05c129/latest/ encountered an error during installation. readTextFromPath utils.ts:65 extractFromSrc pyconfig.ts:50 loadConfigFromElement pyconfig.ts:35 loadConfig main.ts:106 _realMain main.ts:86 main main.ts:68 pyscript main.ts:289 <anonymous> pyscript.js:27252 (edited)

nascif

https://6fd75ce3-df0a-4384-b255-195e41c4f02f.pyscriptapps.com/0201e04e-18e2-42f7-84ed-8760af05c129/latest/ "Service Worker Example"

I see an error around the missing file for the Service Worker ... are you sure all files are there?

A few gripes/bugs for https://pyscript.com:

For a while now, the "Trending" and "Featured" scripts on https://pyscript.com are mostly broken. Right now, there is no user named "examples", for instance.
When you create a new script, it creates an index.html pointing to releases/2023.05.1, not the latest version, which I would expect.
I see no examples for MicroPython
Is "Trending" real at all? All four scripts mentioned there have not changed in a while and are all from March 11

It would be nice to some TLC applied to the site.

@chris.laffra We are in the process of transferring the ownership of the @examples account on pyscript.com In fact, we removed the username from the old account this Friday only. The idea is to start fresh due to the recent release of PyScript Next i.e. 2023.11.1 which is a complete overhaul and re-write with a new core... The new project that points to 2023.05.1 is yet to be changed. Examples for MicroPython will also be added to the above mentioned @examples account. Trending is not really real for now, but is on the roadmap.

madhurt

@chris.laffra We are in the process of transferring the ownership of the @examples account on pyscript.com In fact, we removed the username from the old account this Friday only. The idea is to start fresh due to the recent release of PyScript Next i.e. 2023.11.1 which is a complete overhaul and re-write with a new core... The new project that points to 2023.05.1 is yet to be changed. Examples for MicroPython will also be added to the above mentioned @examples account. Trending is not really real for now, but is on the roadmap.

Good to hear all these updates to pyscript.com are on the roadmap! Looking forward to the upcoming changes.

Looking forward to MicroPython examples. I have this code that doesn't work on Micro "mpy" (button is grey and inactive), but works perfectly in "py" <button id ="btngetword" py-click="getword">Get New Word</button> <script type="mpy" src="./main.py" .... def getword(event): .... (edited)

MikeM

Looking forward to MicroPython examples. I have this code that doesn't work on Micro "mpy" (button is grey and inactive), but works perfectly in "py" <button id ="btngetword" py-click="getword">Get New Word</button> <script type="mpy" src="./main.py" .... def getword(event): .... (edited)

I think the solution there may be changing py-click to mpy-click

BRILLIANT!! Thanks 39AndABit, mpy-click did the trick!!!

2

Problem with random when in MicroPython: It produces a stream of pseudo random numbers, from eg. random.randint(0,255), BUT it produces the same stream each time. So it doesn't seem to seed at the start. I can force seed with random.seed(45), but how to generate the integer (here 45 for example)? (I failed miserably in trying to get a system time). I notice the same problem on web with a dice thrower: https://leriomaggio.pyscriptapps.com/pyscript-dice-roller/latest/ (edited)

MikeM

Problem with random when in MicroPython: It produces a stream of pseudo random numbers, from eg. random.randint(0,255), BUT it produces the same stream each time. So it doesn't seem to seed at the start. I can force seed with random.seed(45), but how to generate the integer (here 45 for example)? (I failed miserably in trying to get a system time). I notice the same problem on web with a dice thrower: https://leriomaggio.pyscriptapps.com/pyscript-dice-roller/latest/ (edited)

Have not tried this, but you could use JavaScript's random function:

import js
from random import random

random.seed(js.Math.random()*2**16)

Or something similar

Jeff, thanks for the suggestion, but js.Math.random() seems to just produce a zero. I'll keep trying - otherwise I'll ask user to type in a random number to start the seed.

MikeM

Jeff, thanks for the suggestion, but js.Math.random() seems to just produce a zero. I'll keep trying - otherwise I'll ask user to type in a random number to start the seed.

ouch ... just tried myself and it looks indeed that MicroPython has issues with JS floating numbers ... it's not just Math.random ...

09:18

FWIWI I've just reported to MicroPython discord channel the issue ... it looks like any float 32 number in JS gets casted as integer which is ... well ... unfortunate.

@MikeM maybe using this workaround works?

Object.defineProperty(globalThis, 'seed', {get() {
    const random_val = new Uint8Array(1);
    crypto.getRandomValues(random_val);
    return random_val[0];
}});

and then

import js
from random import random

random.seed(js.seed)

btw ... another possibility is to random.seed(js.Date.now()) as that basically represent the default None in Python where the seed is based on system time ... that should work too, without bothering the crypto namespace.

13:00

a discussion about fixing this ourselves in PyScript has started here https://github.com/pyscript/pyscript/discussions/1883 feel free to chime in

Patching the "patch-able" · pyscript pyscript · Discussion #1883

Coming after this discussion #1880 I think we have the opportunity to smooth out other gotchas here and there before such gotchas land mainstream. The js.document is just a common use case for work...

Many thanks, Andrea, random.seed(js.Date.now()) solved the problem.

2

Andrea Giammarchi

FWIWI I've just reported to MicroPython discord channel the issue ... it looks like any float 32 number in JS gets casted as integer which is ... well ... unfortunate.

This I found out as well. Was quite a surprise and hard to debug

MikeM

Problem with random when in MicroPython: It produces a stream of pseudo random numbers, from eg. random.randint(0,255), BUT it produces the same stream each time. So it doesn't seem to seed at the start. I can force seed with random.seed(45), but how to generate the integer (here 45 for example)? (I failed miserably in trying to get a system time). I notice the same problem on web with a dice thrower: https://leriomaggio.pyscriptapps.com/pyscript-dice-roller/latest/ (edited)

From the docs :

random.seed(n=None, /)
Initialise the random number generator module with the seed n which should be an integer. When no argument (or None) is passed in it will (if supported by the port) initialise the PRNG with a true random number (usually a hardware generated random number).

The None case only works if MICROPY_PY_RANDOM_SEED_INIT_FUNC is enabled by the port, otherwise it raises ValueError.

As no ValueError is raised, it seems its not supported (yet) in Pyscript Duh - I should have read back ahalf a page (edited)

Jos

From the docs :

random.seed(n=None, /)
Initialise the random number generator module with the seed n which should be an integer. When no argument (or None) is passed in it will (if supported by the port) initialise the PRNG with a true random number (usually a hardware generated random number).

The None case only works if MICROPY_PY_RANDOM_SEED_INIT_FUNC is enabled by the port, otherwise it raises ValueError.

As no ValueError is raised, it seems its not supported (yet) in Pyscript Duh - I should have read back ahalf a page (edited)

it's eventually MicroPython, not PyScript, but that (IIRC and AFAIK) has been fixed .. which version are you using?

I think PyOdide or MicroPython or WASM or PyScript, or whatever causes a memory leak. This error happens after reloading my app a hundred times with a worker with pandas and matplotlib inside of it. I then had to restart my Chrome browser to get my app to load okay again. Does the PyScript project have a regression test for browser memory leaks?

chris.laffra

I think PyOdide or MicroPython or WASM or PyScript, or whatever causes a memory leak. This error happens after reloading my app a hundred times with a worker with pandas and matplotlib inside of it. I then had to restart my Chrome browser to get my app to load okay again. Does the PyScript project have a regression test for browser memory leaks?

how would you suggest we do that? bear in mind leaks in specific interpreters are rather specific interpreters issues so the question would probably be "do interpreters have such regression test?" because we run on JS and JS doesn't leak on refresh, it's a fresh new env every single time.

One way to test this would be to use a headless browser, load a document that imports pandas, and run the whole thing 1,000 times and report the memory increase of the process?

14:22

It would be a defensive measure for PyScript to insulate itself from issues in PyOdide or MicroPython. For me it is very hard to do this kind of analysis myself, but it will impact my users down the road. The good-old "blame analysis game" in action. They blame me, you blame PyOdide, they blame Chromium, turtles all the way down.

.

chris.laffra

One way to test this would be to use a headless browser, load a document that imports pandas, and run the whole thing 1,000 times and report the memory increase of the process?

it's a bit unpractical if we don't want CI to run "forever" per each change we do ... and if we detect anything it's still not on us so it's a half-futile measure to take as there's nothing we can do onc ewe detect that ...

There is a difference between unit tests, integration tests, and regression tests.

chris.laffra

There is a difference between unit tests, integration tests, and regression tests.

we run everything on code changes ... when/how should we run regression tests?

Andrea Giammarchi

we run everything on code changes ... when/how should we run regression tests?

and then again ... what would a failing test tell us or what could we do after?

Andrea Giammarchi

and then again ... what would a failing test tell us or what could we do after?

the 1000 times run is also flaky as metric ... maybe it breaks at 1001 ... so I think pyodide and MicroPython should do their checks before releases, at least, or it'll be a too late indirection from our side.

14:43

a concrete deadlock-release example: we have those tests, we want to release, then we run that regression test you are suggesting ... too bad, latest pyodide or MicroPython fail ... tey don't have such test, they think theyve fixed the issue, we re-update dependencies and re-do all the things before release ... oooops, we spot again the regression, we bring this to their attention ... and so on and so forth

The alternative is that PyScript breaks me and I have even less idea what is broken. Right now, the app simply does not load and I have to now build in a JS check to verify PyScript actually loaded. If not, I show a dialog "PyScript did not load somehow, we suggest you restart your browser.". I consider myself a customer of PyScript, not of MicroPython or PyOdide, as I have no idea how to even contact those teams.

@chris.laffra do you have a minimal-ish example of this behavior? Is it as simple as a worker tag that loads Matplotlib and pandas via py-config, and then refreshing the page over and over?

Jeff Glass

@chris.laffra do you have a minimal-ish example of this behavior? Is it as simple as a worker tag that loads Matplotlib and pandas via py-config, and then refreshing the page over and over?

Right now I have not. I have seen this happen a few times now with PySheets after a few days of developing on Chrome. My earlier mention of "1,000" was to suggest how performance regression tests work for memory leaks. It was not to suggest an explicit count, of course. The leak happens for each load, I would bet for $20.

14:57

I am guessing it is a leak, based on close to 40 years of experience with systems like these. But, I am always willing to be surprised

14:58

I will keep an eye on my Chrome instance the coming days to see if I can see a memory increase

Please do! It's surely possible it's in PyScript, or it's upsteam components, or perhaps something in the way any of those are interacting with your particular dev environment... regardless, PySheets as an advanced PyScript application and development process is really helpful for us in general

I am not sure how to see a trend in these numbers without automation, though. This is after a fresh start with one reload of PySheets. So many processes, so much memory

15:03

I only have one tab open. PySheets loads on MicroPython with a PyOdide worker. I have Chrome Devtools open.

chris.laffra

I am not sure how to see a trend in these numbers without automation, though. This is after a fresh start with one reload of PySheets. So many processes, so much memory

Chrome's memory timelines may be a helpful here https://developer.chrome.com/docs/devtools/memory-problems

15:05

Gives some finer grained views of heap memory, though I don't know how practical that is to run over a timespan of days

This is after 10 reloads: (edited)

15:07

Huh... and you're not doing any additional caching on top of PyScript? Just going a quick test with pandas and matplotlib in a worker (just installing, importing and doing print(dir(lib)) for both) I'm not seeing the memory usage grow at all after 10 reloads.

there is a performance tab in devtools with a GC collector ... I'd be curious to know if that changes anything ... but then again, if Jeff can't reproduce it by just loading those packages repeatedly, at least we need a minimal example to reproduce the issue.

I am guessing it is a leak, based on close to 40 years of experience with systems like these.

by no mean I wanted to even discuss your experience there, I just answered from a practical level ... if our interpreters have issues, there's little we can do about it!

I consider myself a customer of PyScript, not of MicroPython or PyOdide, as I have no idea how to even contact those teams.

it's a fari point ... but you also use an Operating System and any of its software you use can crash too ... you can blame the OS all you like, but if the issue is within those softwares instead, there's very little the OS can grant you (blue screen of doom anyone?

)

15:26

to me this issue needs a bit more narrowing down:

is the MicroPython side that causes the problem?
is the worker that does?
what if both main and worker use pyodide, is the problem still there? (this answers maybe first point)
are you sure in your code nowhere there's something able to leak across multiple refreshes? I am not excluding a Chromium bug with WASM neither ... but that's why we should find the culprit and decide after how to prevent that issue to happen again in the future, if we can

In any case Chris, we appreciate the report and the digging! From your images above, it seems like something is going on... whether it's in PyScript, or upstream, or downstream, I suspect we'll learn something that can be helpful to future users of PyScript, as well as someone somewhere in the dependency chain

2

Jeff Glass

In any case Chris, we appreciate the report and the digging! From your images above, it seems like something is going on... whether it's in PyScript, or upstream, or downstream, I suspect we'll learn something that can be helpful to future users of PyScript, as well as someone somewhere in the dependency chain

absolutely, I never wanted to push back on this ... rather trying to understand the best way forward, regressions tests or not!

2

I will keep a close eye on this and when it happens again make sure to analyse the browser resources in detail. I am using Brave for all my normal browsing and only use Chrome during development, so it won't be contaminated by other apps at all, if this error shows up again.

Andrea Giammarchi

absolutely, I never wanted to push back on this ... rather trying to understand the best way forward, regressions tests or not!

Totally understand. I was merely asking if you had any tests, not that I was demanding you have them.

I ran into WebAssembly.Memory error again today. This time, I isolated it to PyOdide, as I only use PyOdide for the main thread and my one worker. I see nothing extraordinary in the memory reported by the Activity Monitor. Still not reproducible, other than following the script: "Have Chris iterate on his app for a day"

When I close the devtools window and reload the browser tab, I get the same error. However, it loads fine when I open a new browser tab to the same local URL, with or without devtools open. This makes me think it is related to some stateful memory attached to the tab itself. Namely, when I manually kill the browser tab process and then reload the page it works fine (which resembles starting a new tab). This works even when the original page had a devtools window open. All this suggests the problem is not related to the devtools window (and process), but limited to the main browser renderer process.

chris.laffra

When I close the devtools window and reload the browser tab, I get the same error. However, it loads fine when I open a new browser tab to the same local URL, with or without devtools open. This makes me think it is related to some stateful memory attached to the tab itself. Namely, when I manually kill the browser tab process and then reload the page it works fine (which resembles starting a new tab). This works even when the original page had a devtools window open. All this suggests the problem is not related to the devtools window (and process), but limited to the main browser renderer process.

I need to ask: are you using utilities such as create_callback_once and create_proxy to attach functions around? If the answer is yes, would you mind to be so kind to test the experimental_create_proxy = "auto" flag in your config.toml (or py-config) after removing those wrappers around and see if you can reproduce the error again? That might help us A LOT to convince Pyodide team that manual orchestration of proxies from users-side is actually failing while automatic usage of the GC internals help

(edited)

Andrea Giammarchi

I need to ask: are you using utilities such as create_callback_once and create_proxy to attach functions around? If the answer is yes, would you mind to be so kind to test the experimental_create_proxy = "auto" flag in your config.toml (or py-config) after removing those wrappers around and see if you can reproduce the error again? That might help us A LOT to convince Pyodide team that manual orchestration of proxies from users-side is actually failing while automatic usage of the GC internals help

(edited)

one more hint ... just for sanity check ... if you don't want to go/try the way I have suggested, in your devtools tabs there is a performance one which has a lovely collect garbage in there and I wonder, if pressed from time to time, if it would still reproduce the same error ... this test would instead hint there is a follow-up bug to land in Chromium repository so that if pressing that still creates the issue, we can exclude Chromium code/engine from this issue.

Andrea Giammarchi

one more hint ... just for sanity check ... if you don't want to go/try the way I have suggested, in your devtools tabs there is a performance one which has a lovely collect garbage in there and I wonder, if pressed from time to time, if it would still reproduce the same error ... this test would instead hint there is a follow-up bug to land in Chromium repository so that if pressing that still creates the issue, we can exclude Chromium code/engine from this issue.

last, but not least, if you have any create_once_callable in the mix, and you are usign raw DOM API, please never forget to pass {"once": true} as third argument of addEventListener, as I think there's a granted leak in that pyodide implementation too ... but never got a chance to demonstrate there is, thank you!

Andrea Giammarchi

I need to ask: are you using utilities such as create_callback_once and create_proxy to attach functions around? If the answer is yes, would you mind to be so kind to test the experimental_create_proxy = "auto" flag in your config.toml (or py-config) after removing those wrappers around and see if you can reproduce the error again? That might help us A LOT to convince Pyodide team that manual orchestration of proxies from users-side is actually failing while automatic usage of the GC internals help

(edited)

Is there documentation for the contents for py-config, such as experimental_create_proxy ? Right now I am getting

Error: This borrowed proxy was automatically destroyed at the end of a function call. Try using create_proxy or create_once_callable.

17:03

Google search was not effective, maybe I have to ask any of generative AIs

chris.laffra

Is there documentation for the contents for py-config, such as experimental_create_proxy ? Right now I am getting

Error: This borrowed proxy was automatically destroyed at the end of a function call. Try using create_proxy or create_once_callable.

you are using 2024.1.3 ... right? it's one of those things we didn't promote too much but the code is there https://github.com/pyscript/polyscript/blob/main/esm/interpreter/pyodide.js#L85-L86 ... if your code comes from a worker though, you shouldn't need it, but if it's on main, that error for functions/callbacks involving JS APIs should not happen, accordingly to our integration tests.

polyscript/esm/interpreter/pyodide.js at main · pyscript/polyscript

Polyscript - PyScript single core to rule them all - pyscript/polyscript

Andrea Giammarchi

you are using 2024.1.3 ... right? it's one of those things we didn't promote too much but the code is there https://github.com/pyscript/polyscript/blob/main/esm/interpreter/pyodide.js#L85-L86 ... if your code comes from a worker though, you shouldn't need it, but if it's on main, that error for functions/callbacks involving JS APIs should not happen, accordingly to our integration tests.

I think I am doing it right... I am getting the errors on the main thread, not in a worker. These are caused by event handlers set up with jquery (using LTK).

20:18

I could use some extra pointers as well how to set pyodide.setDebug(true). Where and when should I do that?

chris.laffra

I think I am doing it right... I am getting the errors on the main thread, not in a worker. These are caused by event handlers set up with jquery (using LTK).

uhm ... too bad it doesn't work out of the box but yes, that looks OK ... now the follow up question: how are you freeing create_proxy callbacks in there, if ever?

I do not as all of them tend to be static and long-lived anyways. As an aside, in MicroPython I do not need to use proxies at all and things work magically. What's the difference?

and do you use ever the create_callable_once utility? that's another potential leak to me as on WASM side it gets freed, on JS side not necessarily if that listener never gets removed ... I am just trying to understand your use-case stack though, where if tons of create_proxy is needed, I can already see how many ways that can fail at cleaning up the GC in the long run ... although a refresh should do, you seem to have issues on the same session (no new tab) though, so maybe I can start checking Pyodide to see if anything rings a bell

It could be leaking proxies, that would make sense.

20:23

But, when I refresh a page, I would expect the WASM session to be fully unloaded anyways...

chris.laffra

I do not as all of them tend to be static and long-lived anyways. As an aside, in MicroPython I do not need to use proxies at all and things work magically. What's the difference?

MicroPython is less paranoid about leaks on the Web while Pyodide is overly explicit about what should be log-lived and what shouldn't ... the flag I've suggested should take the best from both worlds but it's clearly not working in your use case and it's hard for me to reproduce that specific use case that fails ... but I will try with jQuery as borrowed Pyodide library and see if there's anything we can do about it.

Let me try to make a minimal example on pyscript.com

chris.laffra

But, when I refresh a page, I would expect the WASM session to be fully unloaded anyways...

there is a sessionStorage that might be "abused" on Pyodide to fast-path/bootstrap stuff on the same tab ... it's the only primitive I know that persists on same tab but would be new/clear on any new tab but I haven't investigated anough to see if that might be the root cause of your issues there.

20:26

nope, that looks like the only reference after a quick search ... https://github.com/pyodide/pyodide/blob/e24c2c248f244a70290d129b01303f8a2af6b3e8/src/js/environments.ts#L34

pyodide/src/js/environments.ts at e24c2c248f244a70290d129b01303f8a2...

Pyodide is a Python distribution for the browser and Node.js based on WebAssembly - pyodide/pyodide

Andrea Giammarchi

nope, that looks like the only reference after a quick search ... https://github.com/pyodide/pyodide/blob/e24c2c248f244a70290d129b01303f8a2af6b3e8/src/js/environments.ts#L34

but ... the mounting point for native FS seems to be based on navigator.storage which is a session related storage too ... and maybe, just maybe, we're into something here https://github.com/search?q=repo%3Apyodide%2Fpyodide%20mountNativeFS&type=code

Build software better, together

GitHub is where people build software. More than 100 million people use GitHub to discover, fork, and contribute to over 420 million projects.

20:33

https://developer.mozilla.org/en-US/docs/Web/API/StorageManager

StorageManager - Web APIs | MDN

The StorageManager interface of the Storage API provides an interface for managing persistence permissions and estimating available storage. You can get a reference to this interface using either navigator.storage or WorkerNavigator.storage.

20:33

are there many/heavy FS operations in your code? it might help narrowing down the research area for this bug

20:35

I will be looking for answers out there but if you could land that reproducible issue we'd have a starting point ... although this new tab works scenario makes me think there's something coupled with the session that might have nothing to do with create_proxy itself.

Well, I got it working correctly (with auto and no errors) as you suggested here: https://pyscript.com/@laffra/pyodide-create-proxy-demo/latest. Will check and clear my browser caches or something. Weird.

20:40

Let's hope that (any) such leaks are not providing a side-channel information attack vector...

chris.laffra

Let's hope that (any) such leaks are not providing a side-channel information attack vector...

not sure I am following this concern ... which leak and/or which side-channel attack?

09:50

the demo also works for me but maybe you forgot the worker attribute beside terminal? I see clicks logged/appended on the page.

@chris.laffra awesome demo today, as mentioned in the meeting ... looking forward to hear more about what works and what doesn't with the experimental create_proxy = "auto" flag but, most importantly, I'd love to understand your previous comment ... as the primitives behind that flag are all standards based and I am sure nobody specd' leaks in the FinalizationRegistry API.

Andrea Giammarchi

@chris.laffra awesome demo today, as mentioned in the meeting ... looking forward to hear more about what works and what doesn't with the experimental create_proxy = "auto" flag but, most importantly, I'd love to understand your previous comment ... as the primitives behind that flag are all standards based and I am sure nobody specd' leaks in the FinalizationRegistry API.

Let's get on a call, this is more of a high-density meeting we should have, I think.

chris.laffra

Let's get on a call, this is more of a high-density meeting we should have, I think.

happy to discuss this afternoon, hopefully before 6pm CET

Display function prints after taking user inputs. But first it was suppose to display print function and ask for inputs. https://_12.pyscriptapps.com/mute-term/latest/

zobi

Display function prints after taking user inputs. But first it was suppose to display print function and ask for inputs. https://_12.pyscriptapps.com/mute-term/latest/

window.prompt blocks the main thread ... nothing will ever happen in there so if you block the main thread with a prompt you cannot expect much else happening in there ... use a timeout to ask for input and see it working.

1

Every time I run Py-editor it show the an error for the first run. https://www.youtube.com/watch?v=oIK3ARnfbPE

Project

pyscript issue

zobi

Every time I run Py-editor it show the an error for the first run. https://www.youtube.com/watch?v=oIK3ARnfbPE

fixed in 2024.5.2

1

Hi, on this page https://pyscript.recipes/latest/basic/running-code/ , the link for "Once you have PyScript running on your page…" is broken

@Jeff Glass ^^^

1

It looks like Element.classes doesn't appear to work properly w/ the MicroPython runtime. If I create a new element via new_div = pydom.create("div", classes=['test-class']) then new_div.classes just returns an empty list. Doing the same thing w/ the Pyodide runtime works properly (returns ['test-class']). Element.add_class() and .remove_class() work in both cases (I see the classes change in the browser's web inspector), but the results don't show up in .classes w/ MicroPython. I'm using the 2024.6.1 release.

17:23

MicroPython v1.23.0 on 2024-05-31; JS with Emscripten
Type "help()" for more information.
>>> from pyweb import pydom
>>> new_div = pydom.create("div", classes=['test-class'])
>>> new_div.classes
[]
>>> new_div.remove_class('test-class')
<Element object at 33b10>
>>> new_div.classes
[]
>>> new_div.add_class('test-class')
<Element object at 33b10>
>>> new_div.classes
[]
>>>

@jburnett thank you for the feedback. Can you create an issue about this please..? https://github.com/pyscript/pyscript/issues/new/choose This is one for @Fabio

Build software better, together

GitHub is where people build software. More than 100 million people use GitHub to discover, fork, and contribute to over 420 million projects.

@jburnett it should be fixed in teh next release.

1

ntoll

@jburnett thank you for the feedback. Can you create an issue about this please..? https://github.com/pyscript/pyscript/issues/new/choose This is one for @Fabio

Not sure if you still needed me to create a ticket for this, but I finally did: https://github.com/pyscript/pyscript/issues/2097

Element.classes doesn't appear to work properly w/ the MicroPython ...

Checklist I added a descriptive title I searched for other issues and couldn't find a solution or duplication I already searched in Google and didn't find any good information or help What ...

1

jburnett

Not sure if you still needed me to create a ticket for this, but I finally did: https://github.com/pyscript/pyscript/issues/2097

Hiya! This is fixed if you use pyscript.web 2024.7.1 (I've put example. code in the issue).

I dont know if this bug is pyscript or piodide, but when I try to use await commands in JS or Python to import packages, it gives an error saying that await is no longer supported or smth

@HalwaHacker in Python, you don't await to import packages. Not quite sure what you're trying to do. Some example code and a use case would be helpful.

HalwaHacker

I dont know if this bug is pyscript or piodide, but when I try to use await commands in JS or Python to import packages, it gives an error saying that await is no longer supported or smth

you need async attribute when you want top-level await, both in Python (in JS you need type="module" that enables it already) (edited)

....I wonder about just defaulting to async - I'm not sure why the default isn't / shouldn't be that.

ntoll

....I wonder about just defaulting to async - I'm not sure why the default isn't / shouldn't be that.

we already mentioned that and I'd be happy to do the switch ... it requires a few updates here and there and I think it's not mandatory to have this ASAP ... it could be next release. About the incoming release though, I need your eyes here because this is important: https://github.com/pyscript/pyscript/pull/2130 (edited)

Updated Polyscript and added Panel worker test by WebReflection · P...

Description This MR fixes holoviz/panel#7015 by updating polyscript to its latest where changes were made to address that issue. Changes updated polyscript to its latest added remote issue test ca...

1

Done

1

Not sure if this is a bug or expected behavior. The specified interpreter version is not used when i add the worker attribute. I am trying to pin an older version of pyodide and when used with the worker attribute it always loads 0.26.2, but as soon as i remove the worker attribute, it loads the pyodide version i want. https://pyscript.com/@sai1494/pyodide-version/ This example as 2 script tags, one with worker attribute and one without. Both are pinned to 0.23.4. One of them is commented. swap them to test with and without worker attribute. The python code will display the pyodide version loaded. Searched if there is an existing bug report, but did not find any.

@sai please report this as a bug. Thank you!

Pyscript shows wrong number of projects created or cloned for me?

Not sure what you're looking at here. More context please? What do you mean by "PyScript"..? How do I recreate this bug? How do I check the numbers are correct? Thank you! Hint: you can't ever give too much context.

sai

Not sure if this is a bug or expected behavior. The specified interpreter version is not used when i add the worker attribute. I am trying to pin an older version of pyodide and when used with the worker attribute it always loads 0.26.2, but as soon as i remove the worker attribute, it loads the pyodide version i want. https://pyscript.com/@sai1494/pyodide-version/ This example as 2 script tags, one with worker attribute and one without. Both are pinned to 0.23.4. One of them is commented. swap them to test with and without worker attribute. The python code will display the pyodide version loaded. Searched if there is an existing bug report, but did not find any.

Actually, I just did it for you... if you have a GitHub account please follow this bug report: https://github.com/pyscript/pyscript/issues/2145 (edited)

Specified version of interpreter doesn't work when running in a wor...

Checklist I added a descriptive title I searched for other issues and couldn't find a solution or duplication I already searched in Google and didn't find any good information or help What ...

ntoll

Not sure what you're looking at here. More context please? What do you mean by "PyScript"..? How do I recreate this bug? How do I check the numbers are correct? Thank you! Hint: you can't ever give too much context.

Total projects i created are 288(didn't include the deleted once) but it shows 1076.

So this is on PyScript.com, and not anything about PyScript itself? Cc @Martin

07:51

@zobi thanks for the bug report, someone will probably be in touch here with more questions. It'd probably be helpful to have your username on PyScript.com.

ntoll

So this is on PyScript.com, and not anything about PyScript itself? Cc @Martin

Yes its about PyScript.com _12 is my username on PyScript.com

So I don't have 3000 projects after all

1

@ntoll pinging you on this because micropython. In the puepy project there is a possibility of very good reactivity but micropython is missing a small critical piece. Specifically: " MicroPython does not seem to support __setattr__, __getattr__ nor __getattribute__, and those are completely central for the proxy operation" I wonder if you can help ? The github issue is here: https://github.com/kkinder/puepy/discussions/17#discussioncomment-10374441 (edited)

Deep state · kkinder puepy · Discussion #17

Hello Ken Would you be interested in making the state deeply reactive by default? If yes, we could look into integrating my syncx project, with features like: Deep changes detection Support for dat...

Neon22

@ntoll pinging you on this because micropython. In the puepy project there is a possibility of very good reactivity but micropython is missing a small critical piece. Specifically: " MicroPython does not seem to support __setattr__, __getattr__ nor __getattribute__, and those are completely central for the proxy operation" I wonder if you can help ? The github issue is here: https://github.com/kkinder/puepy/discussions/17#discussioncomment-10374441 (edited)

Hi, I'm currently on holiday and away from computers. MicroPython does support those dunder methods (I've used them). Not sure what's going on. Perhaps ping MP folk with an example of the problem..?

Neon22

@ntoll pinging you on this because micropython. In the puepy project there is a possibility of very good reactivity but micropython is missing a small critical piece. Specifically: " MicroPython does not seem to support __setattr__, __getattr__ nor __getattribute__, and those are completely central for the proxy operation" I wonder if you can help ? The github issue is here: https://github.com/kkinder/puepy/discussions/17#discussioncomment-10374441 (edited)

what Nicholas said, we do use __getattr__ in various places without issues

pyscript/pyscript.core/src/stdlib/pyscript/workers.py at main · pys...

Try PyScript: https://pyscript.com Examples: https://tinyurl.com/pyscript-examples Community: https://discord.gg/HxvBtukrg2 - pyscript/pyscript

Andrea Giammarchi

what Nicholas said, we do use __getattr__ in various places without issues

I have initiated a discussion on the micropython webassembly forum as per @ntoll suggestion.

https://github.com/orgs/micropython/discussions/15711

No answers yet

Deep reactivity in pyscript · micropython · Discussion #15711

Some people are trying to implement deep reactivity in pyscript. This is so we can have a reactive UI layer in the browser (similar to say Vue3). They want the pyodide and micropython interpreters ...

@Neon22 In case this helps, PySheets (which uses PyScript with MicroPython for the UI) uses a Model-View architecture for the cells in a spreadsheet. I override setattr to inform listeners that something changed in the model. This works fine on "standard" PyScript releases for me. See https://github.com/PySheets/pysheets/blob/main/src/static/models.py#L184

pysheets/src/static/models.py at main · PySheets/pysheets

Source for PySheets. Contribute to PySheets/pysheets development by creating an account on GitHub.

1

I am trying to display main.py code in py-editor cell by using src=./main.py but py-editor shows empty. Code within script tags displays in py-editor cell but not from src=./main.py.

Since the last two days, I am seeing unexplainable crashes from Chromium when loading a worker. This makes me think that Chrome/Brave released a new version of Chromium that breaks workers. It also appears to be timing-related. As soon as the crash happens, reloads never recover. Also, Devtools is impacted as it does not show anything. I suspect this is a race condition deep down in PyOdide/WASM. I have no way to debug this. All works fine on Firefox and Safari. You can try the failure yourself at https://pysheets.app (create a new sheet, let it finish loading the worker, and then reload a few times). Once the tab crashes, you have to close it as Chromium cashes WASM.

chris.laffra

Since the last two days, I am seeing unexplainable crashes from Chromium when loading a worker. This makes me think that Chrome/Brave released a new version of Chromium that breaks workers. It also appears to be timing-related. As soon as the crash happens, reloads never recover. Also, Devtools is impacted as it does not show anything. I suspect this is a race condition deep down in PyOdide/WASM. I have no way to debug this. All works fine on Firefox and Safari. You can try the failure yourself at https://pysheets.app (create a new sheet, let it finish loading the worker, and then reload a few times). Once the tab crashes, you have to close it as Chromium cashes WASM.

This is a known bug in Chrome - @Andrea Giammarchi has investigated it and reported it and it's been lumped in with a bunch of other similar reports to Chrome devs. They (Chrome) say it'll be fixed in the next release.

11:57

Yes, it's a pain in the neck and completely beyond our control.

11:57

@chris.laffra if you look at the video of Tuesday's community call on our YouTube channel (I think I linked to it in #chat on Tuesday after the call), you'll see Andrea explaining what's going on.

11:58

Also, bravissimo (as always) to @Andrea Giammarchi for being on top of this and making sure the issue is properly investigated and reported. Now we await Chrome developers.

1

@ntoll @Andrea Giammarchi is there any workaround? Also, this is not a bug in Chrome, but in Chromium, as other browsers, such as Brave suffer the same behavior.

Sure s/Chrome/Chromium - the work around is don't use latest Chrome (sorry).

12:04

We're not the only ones hit by this bug

Good to hear

@Andrea Giammarchi Is there a link to this known issue? Also, "don't use latest Chrome" is not simple advice as Chrome and Brave insist on auto-updating to the latest version (for security reasons) and no one has a (simple) way to go back to an older version. This means essentially that this Chromium bug kills my commercial product based on PyScript without anything I can do (other than wait for Chromium to fix things and Chrome/Brave to pick up the latest version). Is there really no hack/workaround we can do on our end? My product is down for days now....

12:17

I tried going back to older versions of PyScript, but so far no luck....

12:21

Actually, when I use PyScript 2023.12.1, the bug is not coming up, but all kinds of others things are now broken as the platform moved along quite a bit

chris.laffra

@ntoll @Andrea Giammarchi is there any workaround? Also, this is not a bug in Chrome, but in Chromium, as other browsers, such as Brave suffer the same behavior.

the workaround that worked in other venues was to have only mpy-editor with eventually a setup but as soon as both mpy on main and mpy-editor have a congif (the mpy-editor explicitly) something goes banana ... this is:

only mpy and mpy-editor seem to be affected, all good apparently on Pyodide
only when there is a config attribute for what I could test ... so maybe having an <mpy-config> around could mitigate the issue
with Pyodide for the time being there is no bug ... which is the thing I don't understand

Hard crashes are extremely hard to debug because they don't provide anything at all in the current devtool/console ... with this example you'll see th ebare minimum implementation of the bug https://pyscript.com/@agiammarchi/chrome-crash/latest?files=main.py,pyscript.toml,index.html and here there's the rabbit hole of related issues and too many developers scratching their heads because 127 had no issue whatsoever, 128 created dozen issues report for various things (not just PyScript) but they said 129 should have it fixed https://issues.chromium.org/issues/356329410#comment9 ... to date we've seen at least 2 patches for Chromium and Chrome but no major version bump ... I do hope this is the week we get that too. (edited)

chris.laffra

Actually, when I use PyScript 2023.12.1, the bug is not coming up, but all kinds of others things are now broken as the platform moved along quite a bit

you could check which version of MicroPython there wouldn't cause the issue (see where the interpreter points) and use that interpreter with latest PyScript to see if it works but, like you said, many other things might be broken.

Andrea Giammarchi

the workaround that worked in other venues was to have only mpy-editor with eventually a setup but as soon as both mpy on main and mpy-editor have a congif (the mpy-editor explicitly) something goes banana ... this is:

only mpy and mpy-editor seem to be affected, all good apparently on Pyodide
only when there is a config attribute for what I could test ... so maybe having an <mpy-config> around could mitigate the issue
with Pyodide for the time being there is no bug ... which is the thing I don't understand

Hard crashes are extremely hard to debug because they don't provide anything at all in the current devtool/console ... with this example you'll see th ebare minimum implementation of the bug https://pyscript.com/@agiammarchi/chrome-crash/latest?files=main.py,pyscript.toml,index.html and here there's the rabbit hole of related issues and too many developers scratching their heads because 127 had no issue whatsoever, 128 created dozen issues report for various things (not just PyScript) but they said 129 should have it fixed https://issues.chromium.org/issues/356329410#comment9 ... to date we've seen at least 2 patches for Chromium and Chrome but no major version bump ... I do hope this is the week we get that too. (edited)

P.S. if you don't see the crash in the ifame you cna try to use its full URL and see the crash right away in any Chromium 128 based browser.

In my case (PySheets), the problem never occurrs if I do not use a worker. This is on mpy and py for the UI, regardless. When the worker tries to load, I see the issue.

13:47

So, my suspicions are centered around worker+PyOdide...

That said, https://chrislaffra.com/ is also broken and that one definitely is only based on MicroPython and does not use a Worker....

chris.laffra

That said, https://chrislaffra.com/ is also broken and that one definitely is only based on MicroPython and does not use a Worker....

it works for me, to be honest

chris.laffra

So, my suspicions are centered around worker+PyOdide...

it's possible it's worker ... all I am trying to say is: it's a known Chromium issue with dozen links (private or public) attached ... others on Twitter/X also complained Chromium broke badly something ... I don't want to think this is what happens during August due too many people on vacation but there is literally nothing we can do on our side, unless somebody tells me what's broken. Every other project of mine based on workers has literally zero issue so there is a chance the WASM + Worker combo creates shenanigans in current Chromium but I do hope that will be fixed sooner than later. This happens when we have only one major browser dictating the future of the Web which is sad ... yet if they fix it I am sure it won't happen again, or at least I hope so, as they have tons of tests to reproduce the issue ... feel free to add your tests though as that can only help them being sure evrything is fine. If it turns out it's us doing something that always worked but not it breaks here or there, we'll do whatever we can and in the fastest possible way to fix the issue. I feel like we should apology for the current state of affairs but honestly it's like a windows bug affecting any software running on it ... we are not really to blame and you trying every previous version up to last year should also demonstrate it's nothing we did badly from one day to another. I hope this is at least sensible to reason about. (edited)

Andrea Giammarchi

it's possible it's worker ... all I am trying to say is: it's a known Chromium issue with dozen links (private or public) attached ... others on Twitter/X also complained Chromium broke badly something ... I don't want to think this is what happens during August due too many people on vacation but there is literally nothing we can do on our side, unless somebody tells me what's broken. Every other project of mine based on workers has literally zero issue so there is a chance the WASM + Worker combo creates shenanigans in current Chromium but I do hope that will be fixed sooner than later. This happens when we have only one major browser dictating the future of the Web which is sad ... yet if they fix it I am sure it won't happen again, or at least I hope so, as they have tons of tests to reproduce the issue ... feel free to add your tests though as that can only help them being sure evrything is fine. If it turns out it's us doing something that always worked but not it breaks here or there, we'll do whatever we can and in the fastest possible way to fix the issue. I feel like we should apology for the current state of affairs but honestly it's like a windows bug affecting any software running on it ... we are not really to blame and you trying every previous version up to last year should also demonstrate it's nothing we did badly from one day to another. I hope this is at least sensible to reason about. (edited)

it's also worth saying that the underlying issue seems to be fixed: https://issues.chromium.org/issues/361611791#comment17

1

These kinds of bugs are so hard to detect and test.

story of my days

1

We should get you an appropriate t-shirt...

P.S. don't trust MS Edge versioning ... they say 130 but that's not the Chromium behind ... and they don't mention it. (edited)

Andrea Giammarchi

P.S. don't trust MS Edge versioning ... they say 130 but that's not the Chromium behind ... and they don't mention it. (edited)

they refer Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36 Edg/130.0.0.0 but on Chromium it's still Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36

I created a HackerNews post, to spread the news.... https://news.ycombinator.com/item?id=41456943

laffra

Chromium crashes due to a Worker/WASM race condition, impacting PyScript

1

@Andrea Giammarchi I downloaded Chrome Canary (based on Chromium v130) and it appears to fix the WASM worker problem introduced in Chromium v128 (released on Sept 3). I have yet to see an "Awe, Snap!" error in the Canary build. Now we just have to wait for this to percolate down to the regular versions of Chrome, Edge, and Brave (which make up 80% of all installed browsers).

1

chris.laffra

@Andrea Giammarchi I downloaded Chrome Canary (based on Chromium v130) and it appears to fix the WASM worker problem introduced in Chromium v128 (released on Sept 3). I have yet to see an "Awe, Snap!" error in the Canary build. Now we just have to wait for this to percolate down to the regular versions of Chrome, Edge, and Brave (which make up 80% of all installed browsers).

this is a wonderful news but I hope 129 would fix it already

I've read they might backport the fix to 128 too ... let's hope for the best (today might be the day ... a weekend with Ahw Snaps not so)

In the meantime, I added a quick test to pysheets.app

1

Hello all! First time posting here, I've just started trying out PyScript and it's very excitng! I am wondering how would I be able to debug issues in mobile devices, does anyone have tips? I have a page with a py-editor on https://christianlb.pyscriptapps.com/playground/v1/. On my PC browsers (Firefox, Brave, Chrome) the editor is able to successfully run the code and print the output just below, while on mobile (iPad and iPhone) the code execution hangs on the "loading wheel" in all mobile browsers I've tested with (Firefox Focus, Brave, Firefox, DuckDuckGo, Safari, Chrome) (edited)

chrizzftd started a thread. 10/09/2024 11:01

Brave, Edge, and Chrome are still on Chromium v128, since September 3. This means https://pysheets.app has been crashing on 70% of the world-wide installed browsers for the last 9 days. This is way beyond being acceptable.

What can we as a community to help the Chromium team to test better for PyScript breakage? I have already switched to use Chrome Canary, to use the daily Chrome builds. Any other ideas? Better test cases? @Andrea Giammarchi @ntoll @Fabio

Chris Laffra

Brave, Edge, and Chrome are still on Chromium v128, since September 3. This means https://pysheets.app has been crashing on 70% of the world-wide installed browsers for the last 9 days. This is way beyond being acceptable.

What can we as a community to help the Chromium team to test better for PyScript breakage? I have already switched to use Chrome Canary, to use the daily Chrome builds. Any other ideas? Better test cases? @Andrea Giammarchi @ntoll @Fabio

we discussed this yesterday during the call and it is my major concern too ... they had P1 severity but probably they want to be sure the bug is really fixed (as they are doing by releasing dev channels) ... agreed so many days with such easy way to crash any tab on a browser are far from an ideal scenario and pretty against the premises of the "non-breaking / stable Web" story they keep telling out there ... my only hope is that provided tests and use cases will be reduced and used from now on to guarantee that kind of issue doesn't come up again but long story short: I don't know what else we could do.

Maybe convince Google Maps or Google Calendar to use PyScript for some feature?

That would make this a P0 severity.

Chris Laffra

Maybe convince Google Maps or Google Calendar to use PyScript for some feature?

That would make this a P0 severity.

or Google Sheets

@Andrea Giammarchi I see what you are doing there...

1

IMPORTANT NEWS:

Brave and Chrome have upgraded to the back-ported Chromium 128.0.6613.138.

PyScript and PySheets work again. after a week of crashes

. Edge is still broken.

‍♂️

Oh man, that was not fun ™️

Chris Laffra

IMPORTANT NEWS:

Brave and Chrome have upgraded to the back-ported Chromium 128.0.6613.138.

PyScript and PySheets work again. after a week of crashes

. Edge is still broken.

‍♂️

Ive just tested on 128.0.6613.137 and see no issues whatsoever anymore ... this page still crashes on Chromium though ... I have no idea why that's behind 128.0.6613.119 on ArchLinux ... this is awkward ...

Andrea Giammarchi

Ive just tested on 128.0.6613.137 and see no issues whatsoever anymore ... this page still crashes on Chromium though ... I have no idea why that's behind 128.0.6613.119 on ArchLinux ... this is awkward ...

That test works fine on my Brave and Chrome (on Windows, they auto-updated overnight). It still fails on Edge. (edited)

Chris Laffra

That test works fine on my Brave and Chrome (on Windows, they auto-updated overnight). It still fails on Edge. (edited)

just forced-refreshed my ArchLinux dbs for updates and Chromium 128.0.6613.137 indeed works without any issue ... well, that's a relief!

11:36

Edge crashes even on Dev channel here on Linux ... bummer!

The versioning in Edge is unexpected as it does not reference to a Chromium build?

Chris Laffra

The versioning in Edge is unexpected as it does not reference to a Chromium build?

not only that, they give the same versioning twice out of navigator.userAgent

11:38

my Edge-Dev states Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36 Edg/130.0.0.0 while in its about it's 130.0.2808.0 ...

Also, it looks like Edge may be broken for another three weeks... https://learn.microsoft.com/en-us/deployedge/microsoft-edge-channels

Microsoft Edge channel overview

let's hope for the best https://x.com/WebReflection/status/1834180923257954756

Andrea Giammarchi

(@WebReflection) on X

any follower that know people at @MSEdgeDev so gentle to tell them both Chrome/ium and Brave shipped with either 128.0.6613.137 or 128.0.6613.138 which fixes a huge crashing issue still present in all Edge channels? THANK YOU

in other news: PyScript is back, no more crashes!

Python runs anywhere ™. For PyScript this means: Write your code once, then test it on Firefox, Chrome, Brave, Safari, Edge, Chrome Canary, Brave Canary, Edge Canary.

11:45

I literally test my code on 8 browsers now

that's the story of any Web project to me ... with or without Python

The first commercial web product I worked on was on IE4 - I still have scars

JS apis, CSS features, canvas shenanigans, or WebGL, WebGPU, SerialPort and others or literally anything that is not 5 years old and already battle-tested

... I wrote polyfills for IE4 earlier in my career, the "best" thing was: no try/catch statement until 5 came out ... it was nearly impossible to test features in a way that wouldn't break everything else ... "fun times"

11:48

in PyScript we have WASM, Atomics, Workers, SharedArrayBuffer, FinalizationRegistry, IndexedDB ... basically we're living on the edge of the platform but I hope these issues won't repeat in the future (edited)

btw, @Chris Laffra accordingly with this related commit there are regression tests so this particular issue won't likely ever happen again https://issues.chromium.org/issues/361611791#comment17 ... see the commit related files and all the tests around, including the top comment with a test itself

1

12:07

all tests visible in here, basically these are the same proposed in the issue https://chromium-review.googlesource.com/c/v8/v8/+/5833740 ... so weird they managed to break async module instantiation from a day to another and took a week+ to fix the issue ... I guess it's good to be sure no async issues around modules would ever happen again.

I lost a week of my live

2

It's been fixed for Brave at least, seemingly.

@Andrea Giammarchi Could it be that Google adopted PyScript for its Android apps and has a frozen Chromium v128 ? https://old.reddit.com/r/pixel_phones/comments/1fh4f8v/google_app_keeps_crashing/

From the pixel_phones community on Reddit

Explore this post and more from the pixel_phones community

1

Chris Laffra

@Andrea Giammarchi Could it be that Google adopted PyScript for its Android apps and has a frozen Chromium v128 ? https://old.reddit.com/r/pixel_phones/comments/1fh4f8v/google_app_keeps_crashing/

what was crashing PyScript had nothing specific to PyScript, it was an async issue with the module evaluator in Chromium

I just tried to use @Jeff Glass recipe again for downloading images. However at some point assign has disappeared from the js API. E.g. js_array.assign(my_stream.getbuffer()) errors with no assign available if I do a dir() I get:

['__class__', 'find', 'join', 'keys', 'map', 'reverse', 'set', 'sort', 'values', 'buffer', 'fill', 'filter', 'new', 'slice', 'reduce', '4', '5', '6', '7', '8', '9', '0', '1', '2', '3', 'length', 'toString']

(edited)

ok. I think its to do with mpy. io.BytesIO doesn't give expected results and neither does Uint8Array Tricky to work out how to do this kind of thing in mpy - guess we need pyodide for more than I thought

@Neon22 hmmm... "it should work" ™️ Would be interested in what you're seeing with MP io.BytesIO

@Andrea Giammarchi - expecting you have an idea here - when I load https://pysheets.app, the worker thread takes a variable amount of time to load. All I do is load a PyOdide worker. Sometimes it takes 3s (from home), but anywhere I have been traveling it talks around 7s, 11s, and even 16s. Where does the variability come from? Shouldn't everything be cached? Why don't I see this variability for main-thread PyOdide and only for workers???? This run is with a warm cache from my hotel room in Copenhagen.... (edited)

20:51

As you can see, I disabled the browser cache in this run. But, even if I turn the cache on and reload the page 2-3 times, the best time I see here in Copenhagen is 11.2s for the worker to load. This is baffling to me...

ntoll

@Neon22 hmmm... "it should work" ™️ Would be interested in what you're seeing with MP io.BytesIO

feel free to clone/play here: https://pyscript.com/@neon22/mpy-master-template/latest?files=main.py,index.html

1

Chris Laffra

@Andrea Giammarchi - expecting you have an idea here - when I load https://pysheets.app, the worker thread takes a variable amount of time to load. All I do is load a PyOdide worker. Sometimes it takes 3s (from home), but anywhere I have been traveling it talks around 7s, 11s, and even 16s. Where does the variability come from? Shouldn't everything be cached? Why don't I see this variability for main-thread PyOdide and only for workers???? This run is with a warm cache from my hotel room in Copenhagen.... (edited)

average of 4 seconds here ... we don't control the Pyodide CDN but we know that we should improve the cache story, it's on our pipe ... Pyodide provides a way to freeze dependencies and it should bootstrap faster after the first time if dependencies didn't change ... that means that we should use our persistent IndexedDB based storage to get previous cache if it matches the current dependencies graph, then use that instead of letting pyimport resolve all the things then store back if it was different and do the same dance next time. This would likely improve repeated refresh/bootstrap time but it might make even slower the initial bootstrap with Pyodide. About main VS worker delay all I can think about is that workers add one more network request so if the connection is unpredictably slow that might add to the equation but otherwise it's exacly the same Pyodide that runs on main so, unless Pyodide does something different in Workers (but it shouldn't in our case) I wouldn't know what else could be the issue.

08:31

pinging @ntoll here as this story should get prioritized somehow (edited)

1

Andrea Giammarchi

average of 4 seconds here ... we don't control the Pyodide CDN but we know that we should improve the cache story, it's on our pipe ... Pyodide provides a way to freeze dependencies and it should bootstrap faster after the first time if dependencies didn't change ... that means that we should use our persistent IndexedDB based storage to get previous cache if it matches the current dependencies graph, then use that instead of letting pyimport resolve all the things then store back if it was different and do the same dance next time. This would likely improve repeated refresh/bootstrap time but it might make even slower the initial bootstrap with Pyodide. About main VS worker delay all I can think about is that workers add one more network request so if the connection is unpredictably slow that might add to the equation but otherwise it's exacly the same Pyodide that runs on main so, unless Pyodide does something different in Workers (but it shouldn't in our case) I wouldn't know what else could be the issue.

also pinging @hood here in case he has some deeper knowledge around Pyodide internals and possible reasons it might be slower or different from a worker ... Hood already mentioned we should be a better job with the caching story which is why I both agree and think it's about the time. There was some blocker though and I am not sure about the current state of affairs there: https://github.com/pyscript/pyscript/pull/2065#issuecomment-2180704295

Provide index_urls via config to override the default. by amjith · ...

Description Add a config option to optionally provide index_urls that can override the default python package index url. Changes A new config option called index_urls that can provide a list of URL...

Can we get an option on Pyscript.com projects to save all our files each time we click the green Run button? That would be handy.

Floppypants

Can we get an option on Pyscript.com projects to save all our files each time we click the green Run button? That would be handy.

Maybe ask this in the pyscript-dot-com channel?

Chris Laffra

@Andrea Giammarchi - expecting you have an idea here - when I load https://pysheets.app, the worker thread takes a variable amount of time to load. All I do is load a PyOdide worker. Sometimes it takes 3s (from home), but anywhere I have been traveling it talks around 7s, 11s, and even 16s. Where does the variability come from? Shouldn't everything be cached? Why don't I see this variability for main-thread PyOdide and only for workers???? This run is with a warm cache from my hotel room in Copenhagen.... (edited)

could you maybe try https://cdn.jsdelivr.net/npm/@pyscript/core@0.6.0/dist/core.js instead of the stable/official URL for PyScript and tell me how that went? I have just enabled Pyodide lockFileURL to avoid re-importing same packages all over the place every single time and I believe after your second run things should be predictably faster than before, thank you! (edited)

15:12

second run because once you use that latest from npm your micropip packages will be already resolved in terms of complexity and whatnot, so I expect some increase in bootstrap like it was in my tests, where just numpy reduced bootstrap time by 1 second which is everything but irrelevant gain here

15:15

insider-hint ... if you add this script on top of your page that includes pyodide at any point in time, you should also see different results from the first run and any consecutive run after:

addEventListener('py:progress', ({ detail }) => {
  const [action, what] = detail.split(/\s+/);
  if (action === 'Loading') console.time(what);
  else console.timeEnd(what);
});

put this as a script before anything else and please let me know how it goes, thanks!

Andrea Giammarchi

could you maybe try https://cdn.jsdelivr.net/npm/@pyscript/core@0.6.0/dist/core.js instead of the stable/official URL for PyScript and tell me how that went? I have just enabled Pyodide lockFileURL to avoid re-importing same packages all over the place every single time and I believe after your second run things should be predictably faster than before, thank you! (edited)

15:29

7-8s seems the average at the moment (from Denmark, over local conference wi-fi data). (edited)

Chris Laffra

Click to see attachment 🖼️

now ... if you have a py-config in there ... could you be so kind to add packages_cache = "never" and tell me, right after, what is the result, assuming you are with the same hardware and internet connection?

I had a new record just now of 5s, so overall it feels much snappier...

1

you can spot the difference by adding, or removing, packages_cache = "never" ... when it's there, it bypasses the caching entirely, when it's not there, or it's not never, first time it re-does the thing then next time you should see the improvements.

15:34

of course this should be documented in our pages once it's confirmed to work as expected /cc @ntoll

LOL. That "never" crashed Chrome

so ... it's not supposed to be used in the wild, it's a last-resource escape hatch to clean up the cache ... now, I could test stuff forever without a crash, so I am more interested in the when it doesn't crash Chrome/ium scenario

I am seeing a consistent 6s start time for my worker, no variability.

with or without the packages_cache = "never" ?

15:39

if interested, this might be expected accordingly to my comment: https://github.com/pyscript/pyscript/pull/2205#issuecomment-2388783992

Updated Polyscript to provide out of the box Pyodide cache by WebRe...

Description This MR brings in latest Polyscript so that Pyodide automatically caches repeatedly needed packages and the bootstrap dance is overly-improved. Published on npm as: JS https://cdn.jsde...

without the "never" line, I did not see any difference, other than Chromium crashing once

15:41

Switching to the cdn definitely helped. On conference wi-fi, I am now getting 5s startup consistently. This is faster than I ever saw it, even from my home, which has really fast Internet (750Mbps)

Chris Laffra

without the "never" line, I did not see any difference, other than Chromium crashing once

I would not focus on that ... but the fact you have a faster connection might answer already that the bottleneck is not entirely into packages resolutions, as these packages need to be resolved and downloaded anyway, the lockFileURL story is just about resolving dependencies, although I got confirmed IndexedDB is not blob constrained so from a runtime URL it should attach itself to the origin site, the original page that created it. The never you added after switching to latest PyScript on npm also means it was working before, or it's working now, because if you don't have that, and you are using the PyScript I've suggested, it means it's working ... your first screenshot demonstrated it's working, now is there where you see improvements? I'll show two different scenarios in a bit (edited)