Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: V2 chart executor direct memory size #18

Closed
wants to merge 34 commits into from
Closed

fix: V2 chart executor direct memory size #18

wants to merge 34 commits into from

Conversation

gpdenny
Copy link

@gpdenny gpdenny commented May 28, 2021

Description:
Bugfix, non-breaking change which fixes an issue.
Default DREMIO_MAX_DIRECT_MEMORY_SIZE_MB not applied correctly.

Test

# Dremio Executor
executor:
  # CPU & Memory
  # Memory allocated to each executor, expressed in MB.
  # CPU allocated to each executor, expressed in CPU cores.
  cpu: 2
  memory: 6144

Expected Behaviour:
Selection_990
(https://docs.dremio.com/deployment/memory-config.html)
DREMIO_MAX_HEAP_MEMORY_SIZE_MB is 4096 and DREMIO_MAX_DIRECT_MEMORY_SIZE_MB is 2048.

Actual Behaviour:
DREMIO_MAX_DIRECT_MEMORY_SIZE_MB is 0 for executors.
This causes the Dremio executor to fail on startup.

    env:                                                                                                                                
    - name: DREMIO_MAX_HEAP_MEMORY_SIZE_MB                                                                                              
        value: "4096"                                                                                                                     
    - name: DREMIO_MAX_DIRECT_MEMORY_SIZE_MB                                                                                            
        value: "0"           
dremio-executor 2021-05-28 11:43:15,838 [main] ERROR ROOT - Dremio is exiting. Failure while starting services.                   
dremio-executor java.lang.NoClassDefFoundError: Could not initialize class io.netty.buffer.PooledByteBufAllocator                 
dremio-executor     at org.apache.arrow.memory.RootAllocatorFactory.checkPoolConfiguration(RootAllocatorFactory.java:83)          
dremio-executor     at org.apache.arrow.memory.RootAllocatorFactory.newRoot(RootAllocatorFactory.java:78)
dremio-executor     at org.apache.arrow.memory.RootAllocatorFactory.newRoot(RootAllocatorFactory.java:56)
dremio-executor     at com.dremio.exec.server.BootStrapContext.<init>(BootStrapContext.java:66)          
dremio-executor     at com.dremio.dac.daemon.DACDaemonModule.bootstrap(DACDaemonModule.java:261)         
dremio-executor     at com.dremio.dac.daemon.DACDaemon.<init>(DACDaemon.java:185)                        
dremio-executor     at com.dremio.dac.daemon.DACDaemon.newDremioDaemon(DACDaemon.java:313)               
dremio-executor     at com.dremio.dac.daemon.DACDaemon.newDremioDaemon(DACDaemon.java:321)               
dremio-executor     at com.dremio.dac.daemon.DremioDaemon.main(DremioDaemon.java:145)  

I think the intended behavior was to subtract the heap memory size (4096 based on 6144 le engineMemory), so direct uses the remaining?

Nirmalya Sen and others added 30 commits April 1, 2019 11:43
Make running it conditional - either this is running or the Dremio cluster.
Controlled via helm value --set DremioAdmin=true

Change-Id: Ia6d30bcda29e5ea606f57aa6ac22b65aad8a3a91
Change-Id: Ica602f61421506708689bedb36d0d30c887c8407
Enabling support of uploads to S3 or ADLS via
config entries in values.yaml.

Change-Id: Ie3233cb7085268ecd6aef010be5f61ad67cefc3a
Change-Id: I3b6746fb8eddeff8c56ae4876453d47bfc30ef9b
Change-Id: I2e3444eccd95cdce7fab18045b43270cbbb07f79
Change-Id: I3534f554ee2168ee88e7ae19380e33696a27f2a3
Change-Id: I83a272457e977ea54dc35812a6d28b68dda666ee
1. Pull in the latest config files
2. Update memory, cpu in values.yaml so that they work with the
recommended machine types in the docs

Change-Id: I47970acda6fc776171f1970ddacb7e0d78dbff58
Plus change REAME layout and two label changes.

Change-Id: Ia5e05f13910db5c03361c6166411704b40bf4a8e
DX-16619: Fix the syntax in values.yaml

Statefulsets attribute is updateStrategy and the default for that
is RollingUpdate. So, deleting those lines would be sufficient
and not loose any functionality.

Kubernetes client libraries to v1.14 seems to be more strict than
earlier version and throws an error with those lines in there.

Dropping the lines works with earlier versions of helm.

Change-Id: I727adc50a883a4801e737aec0be6b84788f6f357
Change-Id: Ic0aa8058ffbcd46144e451621312554b72dc4a67
Certbot uses port 80 to handshake with LetsEncrypt to
generate SSL certificates. So, port 80 is being
opened up by default.

Change-Id: I565bd37b6b2657c67751efd1eee2f4cc45be4ba8
Azure VMs do not automatically do reverse DNS lookup of the VM's
public ip address. Reverse DNS needs to be specifically enabled.

Change-Id: Ie92fe9ca6c6358a3704947f201dae4e209602c66
1. The URL for path.dist values for S3, ADLS are different from
pre-3.2.0 and 3.2.0+. Handle it.
2. Add support for Azure Storage v2 for path.dist

Change-Id: I7730ed4caac22240e579e41641e720bdedf05ec0
Create a Azure Storage V2 account, create a filesystem and
directories in there and configure dremio to use that for
accelerator and uploads.

Change-Id: Ic1ce87d3ed8f2bfc6ac5a8d822af75f742b61d1e
Change-Id: I17e70a423356849c0d2368648a72241b97d5afeb
The image includes useful cloud tools. See the
Dockerfile for the list of tools/clients installed.

Change-Id: I6edd554ffc37b353813b8c351a9d505332267930
Change-Id: I90e2515ebc230fa8da5940498516f9ae313a019e
Change-Id: If8b01fcc3b7d71563a07f8115ce0c4ae03bd3f9a
- Additionally fixes deploying multiple instances of the CFT in the same
  subnet by removing the GroupName and allowing AWS to dynamically
  generate one since GroupName(s) must be unique in a subnet.

Change-Id: I1fb6101528b857a40e09f41aebd11a9843ecb257
- Dremio 4.0.0 or later required.
- Adds the concept of an imageTag to expose features that are
  introduced only in newer versions of Dremio.
- Removes the dremioVersion value that needs to be manually set
  to reference the same version that is used by the image.
- Adds optional Cloud Cache support.  Dist is split between PDFS and
  cloud storage.

Change-Id: Idccb0acbd82a8f3ed50b249599c0b04e93641406
This reverts commit d199978.

Change-Id: If484452e8608f2dbdfa2713ca33bb13b4c92c4f0
- Dremio 4.0.0 or later required.
- Adds the concept of an imageTag to expose features that are
  introduced only in newer versions of Dremio.
- Removes the dremioVersion value that needs to be manually set
  to reference the same version that is used by the image.
- Adds optional Cloud Cache support.  Dist is split between PDFS and
  cloud storage.

Change-Id: I645c53bb772c0d52362052ef77925c08b30cc494
- Updates the ARM template to use an updated base CentOS image.

Change-Id: Id77a3e2bf3918ce1314aba435c66b0830a7a024a
Change-Id: I4ea8b85a712047bcd4f6520da3c1ec5452d8b849
Change-Id: I36a84dd6139030f1860b68c870ee2a7e9743dca3
Change-Id: I7cb9bd86c165bf59654ba2c2a26c173d31e3c01d
- Fixes the context used for extraInitContainers when templating the
  user provided value.

Change-Id: I866716fa636ac6c50971b623f314a6de79fbee69
- Exposed Flight endpoint (default port 32010).
- Added TLS configuration support to the Flight endpoint.
- Added documentation for Flight.

Change-Id: I155543d55e8d78c8b13f244918c5a5f8438d89fe
- Fixes the ability to set annotations/labels on StatefulSets in the
  Helm chart.

Change-Id: Ie21980dbd3b626f434c68c517e7a46a6b16450e5
ryantse and others added 4 commits May 7, 2021 16:20
- Document the recommended approach for creating a custom Dremio image.

Change-Id: I22eebed0e7960f4a8a88520d71576f246675ad65
Upgrade tasks may need access to the Dremio configuration to properly
work.

Change-Id: I65cd1c40e91c12e3748d9f1048c9d3caa0990b11
@gpdenny
Copy link
Author

gpdenny commented Sep 15, 2021

Just a reminder this is here 🤷‍♂️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
4 participants