Good suggestion but my working environment is gcloud. Client will not prefer to buy out licenses or install external tools due to data privacy.

You can write your own code. In bashoperator, you add your cosigo call in scala.

You can write your own code. In bashoperator, you add your cosigo call in scala.

Code*

You can write your own code. In bashoperator, you add your cosigo call in scala.

Ok let me try

Hi, have anyone tried pubsub in GCP

handoop or hive give a free cluster?

👍

i need a cluster with low latency

google colaboratory have a free services for this!!1

i need a cluster with low latency

This group is dead

Someone knows how to ignore the b' and /n' from text file in Spark?

The original string are between single quotes and this letter b or n are in front?

The original string are between single quotes and this letter b or n are in front?

It's Json input b'{date:value, prod:value, price:value}\n'

It's coming that way from the application that we listen

It was just json before

Hii

How to learning hadoop ecosystem ?

How to learning hadoop ecosystem ?

You can start with Definitive Guide

hi

Help me, please

Distcp Between kerberized and kerberized Cluster

Why does this group exist while no one ever talk?

Why does this group exist while no one ever talk?

Criticall mass of members is not reached.

Let me start a discussion: what is your analytics stack?

Let me start a discussion: what is your analytics stack?

Google big query, metabase, max BI.

Hay dataengineers!

Hi, guys, any idea how to convert a Stanford Snap graph into a json ?

Hi, guys, any idea how to convert a Stanford Snap graph into a json ?

You may just load it and create a JSON and save to file

Polynote Notebook Jeremyrsmith.github.io/polynote-2019-slides

Hello all, I am a newbie and trying to learn as much as I can in order to work as a Data Engineer

Hi, Is there any Data Infrastructure Engineer in the United States looking for job change?

Hey can anybody tell me how companies with mature data science organizations use jupyter? Is Jupyter used on the notebooks of the employees directly or is it hosted in a secure infrastructure? I guess the latter can be used to facilitate controlled access to data stores that contain critical personal information so my guess is the latter but I really have no idea how it's typically used

Hi all

I dont know if its ok to post a link to group about apache spark here.

So. If you are interested in learning Apache Spark you can join https://t.me/apache_spark . We have a group to discuss about spark. From time to time we have online meetups in order to discuss about spark internals. Next time we will discuss about groupBy internals

Do you have a Russian group too?

No, i dont .

Maybe you could create one?

ʕ•ᴥ•ʔ

Good idea. But there is a limit for groups number per 1 person in telegram. I reached the limit ).

wow

never heard about that

so sad

I dont remember execly. May be one person can create 10 groups. Something like that. I aslo created @apache_kafka , as examle )

delete it and create the new one!

just kidding

)))

what if i create a group, gives you admin and then i leave the group?

there is a change owner button. let's hack the system

Why you do not want to own the group ?)

I don't know how to promote is, frankly.

Do you have a Russian group too?

@moscowspark

So. If you are interested in learning Apache Spark you can join https://t.me/apache_spark . We have a group to discuss about spark. From time to time we have online meetups in order to discuss about spark internals. Next time we will discuss about groupBy internals

Very great.

@moscowspark

thank you!

Может кому-то интересно будет https://career.habr.com/vacancies/1000059660

Is anyone interested in participating in profitable startup?

Is anyone interested in participating in profitable startup?

who is not?

who is not?

Many

Many

Maybe they don't see it as profitable

Maybe they don't see it as profitable

they interested in stable fixed income :)

The chance of being a crypto scam is huge today

Well we are on market for 2 years as quant trading startup :) Yes we trade on crypto but we do not even a fund, we trade on investor`s accounts using API and have no access to their funds

where can i find a student of stanford university Data Science?

where can i find a student of stanford university Data Science?

On LinkedIn

@dataeng

I have a question on Rstudio

I have a function: plot(wt~mpg,data=mtcars, ...). I want to ask "wt" is independence variable or dependence variable?

Anyone has a group for algorithms and data structures?

Get courses from famous universities at low price join our channel for more updates and offers from Coursera and Udemy. https://t.me/coursera_coursers

I have image data as binary in Hive table. I would like to view the image by retrieving from hive table. How to do that ?

Binary or base64?

https://medium.com/@tomergarber/apache-nifi-the-iops-problem-2974d21eb67c

We are looking for Big Data - Aws engineers and Data engineers with azure data bricks let me know if you have any reference for this

We are looking for Big Data - Aws engineers and Data engineers with azure data bricks let me know if you have any reference for this with 3+yrs of experience Write to apreddy@nisum.com Note: This position is work from anywhere ( Remote Work)

Hi there I need some help and hope and someone from this chat could help me. I've pulled this repo to play around hadoop and hdfs. Everything works fine only one think confuse me. Hadoop generates randomly hostname for datanodes which are not accessible outside of the containers. They only could accessed from within the hadoop docker container, seems that hadoop uses some internal dns. Here is how it looks like. Is there any way to specify the http address statically? Thanks in advance.

Hello All, Am hiring for below positions for Hyderabad location 1.BigData Engineer with aws 2.DataEngineer/Data analyst 3.DataEngineer-Azure databrick Please share profiles to apreddy@nisum.com

Hi guys! Sorry for my english.

I try Connect hive kerberized on ubuntu

Error creating login context using ticket cache: Unable to obtain Principal Name for authentication .

This conf works on w10 and macos