-
Notifications
You must be signed in to change notification settings - Fork 1.1k
ValueError: Big-Endian buffer not supported on little-endian compiler #920
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Thanks for digging into this @kahemker. Can you also post the versions of netcdf and hdf5 that you're using? @dopplershift, do you know anything about the byte-order behavior for netcdf subset on THREDDS? This issue and the SO post linked to above suggest that the returned data is sometimes big-endian and sometimes little-endian. This causes a problem when trying to convert to a |
I am using netcdf4 version 1.5.3 and I am not using hdf5. It does not appear in the setup.py list of dependencies for pvlib-python (required or optional). Here is a pastebin that has my entire conda environment.yml file. |
I'm pretty sure netCDF is supposed to be dealing with this. @WardF can you shed any light on netCDF endian handling? Can someone point to the data in question? |
NetCDF handles endianness under the hood transparently; however, we are dealing with an issue currently (see Unidata/netcdf-c#1338) where there appears to be a problem. We recently got some Big Endian hardware in-house, to debug this for our upcoming 4.7.4 release. |
Describe the bug
Sometimes when using the
process_data
function inforecast.py
(usually by invokingget_processed_data
), the script would crash with the error:ValueError: Big-Endian buffer not supported on little-endian compiler
I found the bug by using
forecast.get_processed_data()
function. I can pull forecast data consistently without errors using the only theget_data()
functions.ValueError: Big-endian buffer not supported on little-endian compiler
To Reproduce
I creating this bug on Windows 10-64 Bit OS with Python 3.8.2. You can follow the script written in this StackOverflow post and occasionally you should see this error.
Expected behavior
Data retrieved should not create runtime errors during processing
Versions:
pvlib.0.7.1
pandas.1.0.1
python: 3.8.2
Additional context
I believe I can fix this bug. I pushed my modified
forecast.py
here.The text was updated successfully, but these errors were encountered: