Apache Spark and In-memory Hadoop File System (IGFS)

IGFS is Apache Ignite’s in-memory distributed file system, which is the Hadoop File System compatible.

<bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
. . . . . . . . .
<property name="fileSystemConfiguration">
<bean class="org.apache.ignite.configuration.FileSystemConfiguration">
<!-- Distinguished file system name. -->
<property name="name" value="myFileSystem" />
<property name="ipcEndpointConfiguration">
<bean class="org.apache.ignite.igfs.IgfsIpcEndpointConfiguration">
<property name="type" value="TCP"/>
<property name="host" value=""/>
<!--<property name="port" value="12345"/>-->
. . . . . . . . .
sparkConf.set("fs.igfs.impl", "org.apache.ignite.hadoop.fs.v1.IgniteHadoopFileSystem");



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Amar Gajbhiye

Technology Enthusiast | Big Data Developer | Amateur Cricketer | Technical Lead Engineer @ eQ Technologic | https://www.bugdbug.com